AI
Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 06:07:37 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 30/09/2026, 17:18:24 UTC.
Briefings
A study reveals that chunked KV-cache compression introduces systematic phase sensitivity in transformers, causing long-context retrieval accuracy to vary by up to 40 percentage points across different compression phases. Researchers pretrained transformers across multiple compression designs and used causal interventions to identify phase specialization, where specific attention components contribute asymmetrically to retrieval at different source phases. This finding suggests that average benchmark scores can mask periodic weak spots in model performance. The study provides a framework for evaluating models with compression by measuring across phases rather than relying on aggregate metrics.
Key points
- Chunked KV-cache compression introduces a new positional coordinate called 'phase,' representing a token's position relative to compression-window boundaries.
- Long-context retrieval accuracy can differ by up to 40 percentage points across phases in large open-weight models, revealing periodic weak spots hidden in average scores.
Context and limitations
The study covers only transformers pretrained with chunked KV-cache compression; results may not generalize to other compression methods or architectures. Phase sensitivity is observed in open-weight models but specific model architectures are not fully characterized. The research is based on causal interventions and idealized models, so real-world performance may vary. The findings are from a preprint and are subject to further validation.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 06:07:37 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeiccxsuollirsnmmevgodtgmuto6p4s2qetpzyhwmd63kt4n7mmiaq