AI

When Retrieval Is Not Enough: Obsolete Facts in Agent Memory

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 28/10/2026, 04:03:25 UTC.

Briefings

Research on MemoryAgentBench shows that retrieving the correct fact is insufficient when obsolete competing statements exist. Using Qwen2.5-7B-Instruct on 6K history, accuracy dropped from 45% to 32% as retrieved evidence budget increased from 64 to 512 tokens. A frozen prompt with equal four-statement evidence size reduced accuracy to 44.6% for Qwen2.5 and 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale. A 32K-history replication favored BM25-64 over BM25-512 (69% versus 58%).

Key points

• Qwen2.5-7B-Instruct accuracy dropped from 45% to 32% on single-hop FactConsolidation when retrieved evidence budget grew from 64 to 512 tokens.
• Replacing one unrelated statement with an obsolete competing statement reduced accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B.

Context

Memory-agent benchmarks often collapse retrieval and answer generation into one score, making it hard to distinguish between missing an update and not using an already retrieved update. This study examines this distinction on the single-hop FactConsolidation task of MemoryAgentBench.

Limitations

Results are based on a single preprint (arXiv:2610.36308v1) and are subject to future revision. The study uses a frozen prompt and limited history sizes (6K, 32K), and the 50 harmful paired flips are classified as stale without independent verification.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 28/10/2026, 04:57:36 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeicgqqtk56ckyksyddyfafjy6kp3ke7vboy2adejfljrmnwzcunpum