AI
Softmax Reparameterization for Output-Head Quantization
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 24/09/2026, 20:00:00 UTC
Briefing created: 28/09/2026, 21:17:13 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 28/09/2026, 14:18:24 UTC.
Briefings
A 2026 study proposes softmax reparameterization as a post-training technique to reduce inference costs in small language models by selecting an equivalent output head before quantization. The method subtracts a scalar multiple of the vocabulary-row mean from every output row, optimizing the selection coefficient via validation KL divergence across three loss functions. Results show significant gains on Phi-4-mini, with AW-MSE KL dropping from 0.936 to 0.256, while benefits extend to C4 and OpenWebMath. The approach preserves full-precision softmax distributions and maintains trained decoders, with a rank-one correction handling nonlinear logit paths.
Key points
- Softmax reparameterization selects a functionally equivalent output head before quantization to reduce inference costs in small language models.
- On Phi-4-mini, AW-MSE KL falls from 0.936 to 0.256, demonstrating significant gains across seven heads where baseline quantization distorts predictions.
Context and limitations
Coverage is limited to seven heads and four quantizers; results may vary with different model architectures or quantization schemes.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 28/09/2026, 21:17:13 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeieil4bkptahlrshyn5ilfwes3ptal7oyqsfw4nfo55ktnn3q6rszy