AI

Real Long-Term Memory for AI: A 50-Million-Token Window That Is Faster and Cheaper Than Recompute

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 09/10/2026, 00:18:24 UTC.

Briefings

A 50-million-token window memory layer named galahad-kv enables AI models to load previously computed KV states from encrypted local NVMe storage, bypassing expensive recomputation. Tests on Gemma 4 12B and 31B models showed 82% to 98% accuracy on factual questions, with loading times 2.8x to 4.3x faster than recomputation and 8.8x to 12.3x less GPU energy. The system maintains flat GPU memory usage across the entire 50M-token stream. This approach uses public software and a free license for the package.

Key points

• The galahad-kv memory layer saves KV states of 16,000-token blocks to encrypted local NVMe disk, allowing byte-exact loading without recomputation.
• Accuracy on factual questions reached 82% for the 12B model and 98% for the 31B model, with no model generating fabricated answers.

Context

Large language models typically recompute internal key-value states for every prompt, which is costly. The galahad-kv package provides a solution by storing KV states in encrypted local NVMe disk, enabling faster and cheaper inference for massive token windows.

Limitations

This is a single-GPU test using public software and a free license. The system uses terabytes of local NVMe disk for storage. Accuracy depends on the model's ability to answer questions, not the window size. The test protocol specifically resists common gaming techniques for long-context benchmarks.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 13/11/2026, 21:30:28 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeifetvarivvwpqplyqu6ijwlwl3jpcqn2ojnoikddrmkmmweailu4e