AI

SWE-Game: Can Coding Agents Build the Games We Want?

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 08/10/2026, 04:18:24 UTC.

Briefings

SWE-Game evaluates 247 tasks using 41 Godot games across 13 categories. Opus5 scored highest overall, with Best overall scores below 60/100. Human-labeled checks achieved 92.59% balanced accuracy, significantly outperforming video-based VLM judges at 78.41%. Rubric-based visual scores correlated strongly (0.829) with human ratings. Results characterize current agent capabilities in game development.

Key points

• Opus5 achieved the highest overall score in all five task types within the SWE-Game benchmark.
• Human-labeled executable checks achieved 92.59% balanced accuracy, significantly outperforming video-based VLM judges at 78.41%.

Context

SWE-Game is a benchmark of 247 tasks grounded in 41 executable reference Godot games spanning 13 gameplay categories in 2D and 3D. Evaluation combines engine-state checks, certified reference-input replay, and agent-authored feature demonstrations to assess mechanic correctness, demonstrated playability, and behavioral restoration and preservation after repairs. Rubric-based visual scores separately assess presentation. Results characterize current agent capabilities across game-development activities.

Limitations

Evidence is limited to 247 tasks and 41 games; evaluation combines engine-state checks, certified reference-input replay, and agent-authored feature demonstrations. Human-labeled behaviors from 100 agent-built games were used for behavioral assessment. Rubric-based visual scores correlate with human ratings but do not replace them. No additional background or causal claims are supported beyond the reported metrics.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 04/11/2026, 22:38:53 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeic2penvbz6p7uvphd4phsf5soirr3f5l66lreaxilsysp4bt3ogyu