AI
WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 28/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 09:45:58 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 03:18:25 UTC.
Briefings
WUSH-KV is a low-bit KV-cache quantization method that reduces inference costs by constructing data-aware transforms from second-order statistics of context length and batch size. The method uses calibration data to separate key and value transforms, with the value transform folded into weights and the key transform applied after RoPE. Under mild assumptions, the WUSH transform is near-optimal for the QuEST INT quantizer, achieving the lowest end-to-end perplexity among tested transforms. Evaluated in SGLang with OSCAR-style quantization, WUSH-KV performs comparably to or outperforms OSCAR at 2-bit across various models and tasks.
Key points
- WUSH-KV reduces KV-cache memory and bandwidth costs by adapting a data-aware transform from second-order statistics of context length and batch size.
- The method achieves the lowest end-to-end perplexity among tested transforms by integrating WUSH-KV into SGLang with OSCAR-style percentile-clipped affine quantization.
Context and limitations
Evaluation is limited to SGLang with OSCAR-style quantization; performance comparisons are based on end-to-end perplexity and layerwise reconstruction error under mild assumptions.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 09:45:58 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeihs7ibtmg7frcy4ddzhtk6mesht7wr5z6nhau4iz6b3ja26qlxntm