AI
AOS-Bench: Can LLM Agents Manage Dynamic Workloads?
Publication: cs.AI updates on arXiv.org
Author: Erle Zhu, Genrui Zhang, Keyang Hu, Mingyu Su, Hongning Wang
Source published: 27/10/2026, 04:00:00 UTC
Briefing created: 22/11/2026, 16:54:13 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 28/10/2026, 02:33:24 UTC.
Briefings
AOS-Bench evaluates LLM agents managing dynamic workloads by testing systems under shared CPU and memory constraints. Results show that isolated task success does not carry over to concurrent streams, with heavy workloads causing 13-23 point losses per system. A fixed concurrency of two matches the best agent-managed policy on medium and heavy workloads, keeping median CPU use below 30% of provisioned cores.
Key points
- AOS-Bench introduces a benchmark evaluating LLM agents as workload managers under shared CPU and memory constraints, where verified tasks arrive over time.
- On heavy workloads, isolated task success does not carry over evenly, resulting in 13 to 23 point losses per system under concurrent execution.
Context
This research addresses the shift in coding agents from handling one request at a time to serving as persistent assistants managing streams of work like builds and bug fixes. The benchmark provides a way to measure workload management capabilities that existing benchmarks cannot, as they evaluate tasks in isolation.
Limitations
Evidence is limited to five model-harness pairs across medium and heavy workloads. Results are based on isolated task success metrics rather than stream success. No additional background or context is supplied beyond the abstract.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 22/11/2026, 16:54:13 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeibqwpiknxekjs4hxbgj6bx63trbc3kj5aeoikwhe46bjv4so62tle