AI
AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 24/09/2026, 20:00:00 UTC
Briefing created: 28/09/2026, 04:57:10 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 28/09/2026, 04:18:24 UTC.
Briefings
AgentWorld is an open-source benchmark evaluating long-horizon collaboration in multi-agent LLMs, featuring 100 human-annotated tasks across a MMORPG sandbox. It introduces Causal Collaboration Effectiveness (CCE) to measure effort contribution beyond binary success. Experiments with Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B revealed that even top models achieve only 52.0% task success, with systematic failures like communication breakdowns and role confusion.
Key points
- AgentWorld consists of 100 human-annotated tasks, each with 100 augmented variants, requiring 3-20 agents to coordinate over 50+ interaction rounds in a MMORPG sandbox.
- The benchmark introduces Causal Collaboration Effectiveness (CCE) as a graph-based metric to quantify the fraction of team effort contributing to outcomes, revealing that top models achieve only 52.0% task success.
Context and limitations
The benchmark operates in a blackbox setting where agents lack access to others' internal states, and results are limited to the specific models tested: Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 28/09/2026, 04:57:10 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeibsma5smimrul6ow2csrnmrjlohyahnljm7ulskpa3lly7r6yyzmi