AI
Agent Error Dataset: Scaling 50,000 Error--Diagnosis Pairs for Failure Analysis and Error-Aware Post-Training
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 09:31:09 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 04:18:24 UTC.
Briefings
The Agent Error Dataset (AED) introduces 50,228 error-diagnosis pairs from 9,961 source tasks across 33 environments, enabling cross-setting failure analysis and re-diagnosis without repeating rollouts. A five-stage pipeline collects natural failures, generates diagnoses, and tests corrections against recorded evidence. Across 3,062 matched replay pairs, first-proposal corrections raised verifier pass rates from 18.4% to 51.1%. Using a frozen diagnosis release, full-diagnosis fine-tuning on 1,656 source tasks improved Qwen3-8B's exact-step agreement from 47.2% to 63.6%. Action-only repair training outperformed success-only training on WebShop-lite by 6.67 percentage points.
Key points
- The Agent Error Dataset (AED) comprises 50,228 error-diagnosis pairs from 9,961 source tasks across 33 environments, 19 harness families, and 23 policy models.
- First-proposal corrections raised verifier pass rates from 18.4% to 51.1% across 3,062 matched replay pairs in the AED pipeline.
Context and limitations
The dataset covers 33 environments and 19 harness families but does not specify the exact number of source tasks or policy models used in the full AED collection.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 09:31:09 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeicpdcohghsfofwjp5ccd2evkpyetok5p3caplpvfajelmu45n4vuy