AI
Disaggregated Quantization: Specializing LLM Prefill and Decode
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 21/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 02:32:02 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 28/09/2026, 19:18:24 UTC.
Briefings
Researchers propose disaggregated quantization (DQ) to optimize LLM inference by separating prefill and decode operations. On Qwen 3 and Gemma 3, removing activation quantization during decode improves accuracy on decode-heavy tasks without increasing cost. Training separate compute-native prefill weights accelerates prompt processing while matching or exceeding weight-only inference accuracy at 2-3-bit decode. Released Qwen3.8-27B GGUF decoders show 1-bit accuracy gains of 32.5 points on MMLU-Pro and 35.3 on MMMU-Pro when paired with an NVFP4 prefiller. Offloaded disaggregated prefill (ODP) streams weights from SSD, achieving a 1.78x time-to-first-token speedup over the weight-only baseline at 8K prompt length in llama.cpp.
Key points
- Disaggregated quantization (DQ) separates prefill and decode operations to optimize LLM inference accuracy and speed.
- On Qwen 3 and Gemma 3, removing activation quantization during decode improves accuracy on decode-heavy tasks without increasing inference cost.
Context and limitations
Accuracy improvements are validated on specific models (Qwen 3, Gemma 3, Qwen3.8-27B) and tasks (MMLU-Pro, MMMU-Pro) using vLLM and llama.cpp environments.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 02:32:02 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeichcpg2634f5iomtkwr3aamomvmweugrfuixcpp2rx3263dnpgqg4