AI

KQuant - new quantization technique, 3.7× better quants for Qwen Flash Next 180B, Qwen 27B, 35B, 9B (measured in KL-divergence reduction) (MLX)

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 12/11/2026, 18:47:26 UTC.

Briefings

An ML engineer named Alish from Nace developed KQuant, a new quantization technique for Qwen family models. Using Fable 5.2 and Astra, the team achieved significant KL-divergence reductions across 4B, 9B, 27B, and 35B models. The KQuant Q4 variant for 35B models reduced KL divergence by 3.7x compared to public baseline 4-bit packs, while maintaining comparable or faster inference speeds on M5 Pro hardware. The project is open-sourced on Hugging Face with specific model weights available.

Key points

• KQuant achieved a 3.7x reduction in KL divergence for Qwen3.6-35B-A3B models compared to public baseline 4-bit packs.
• KQuant Q4 quantization for 9B and 27B models reduced KL divergence by 1.5x to 2.6x compared to public baseline 4-bit packs.

Context

Alish, an ML engineer from Nace, led the optimization of quantization for Qwen models using Fable 5.2 and Astra. The work focuses on minimizing KL-divergence for accuracy while reducing model size, with results available for Qwen3.5-4B, Qwen3.5-9B, Qwen3.8-27B, and Qwen3.6-35B-A3B.

Limitations

KL divergence measurements are original-text based and do not represent task-accuracy scores. Public baseline 4-bit packs vary by model (e.g., lmstudio-community for 4B). Inference speed comparisons are averages on M5 Pro hardware and may vary by specific hardware or configuration.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 13/11/2026, 07:38:04 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeihufxe77mmsfhu5dugqtx6ee4y3w2hku5jafvh7kf2fxmrzkjcozq