AI
Draft-KV: Learning Useful Latent Communication Between Language Models
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 10:39:57 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 06:18:25 UTC.
Briefings
Draft-KV enables language models to share internal states via a specialized interface rather than decoded text, achieving 78.04% MMLU-Redux accuracy with a frozen 0.5B receiver. This method outperforms standard message replacement by up to 0.60 points and allows the sharer to be dispensable. The system uses linear projections and gated attention for side memory read-through, with training shifting from message reconstruction to answer supervision.
Key points
- Draft-KV achieves 78.04% MMLU-Redux accuracy using a frozen 0.5B receiver, significantly outperforming the 37.45% accuracy of the same model receiving decoded messages alone.
- Replacing messages with unrelated questions improves accuracy by at most 0.60 points, demonstrating that the interface can supply gains while making the sharable model dispensable.
Context and limitations
Accuracy figures are snapshot-based and limited to five method-dataset pairs; results may vary across different models or tasks.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 10:39:57 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeici4cwgokshz5celk32ucgb2hrv4a5dihy2czfwqsqlb3zeu7af7m