AI
Advancing Quantization for R&R Models with Microscaling Formats
Publication: cs.AI updates on arXiv.org
Author: Yavuz Yetim, Bor-Yiing Su, Zhe Wang, Arthur Liu, Pranav Sharma, Ying Wang, Randy Shuai, Luke Pfister, Ruichao Xiao, Bill Zhao, Zihao Liu, Liyuan Li, Jiangyuan Li, Gaoxiang Liu, Nathan Yan, Geonhwa Jeo
Source published: 16/11/2026, 05:00:00 UTC
Briefing created: 16/11/2026, 05:46:51 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 16/11/2026, 05:03:25 UTC.
Briefings
This paper presents empirical evidence that MXFP6 and MXFP8 quantization formats enable significant speedups in Recommendation and Ranking (R&R) models running on AMD MI350X hardware. The study compares these formats against FP16 and FP8, demonstrating that MXFP6 achieves 1.73x to 1.99x speedups on linear layers while maintaining a 0.01% Normalized Entropy margin relative to baseline models. MXFP8 offers comparable accuracy to row-wise FP8 with up to 2.90x speedups on individual layers, and selective 6-bit quantization yields 7.6% to 14.3% throughput gains over FP8 configurations.
Key points
- MXFP6 quantization achieves 1.73x to 1.99x speedups on linear layers while maintaining a 0.01% Normalized Entropy margin relative to baseline models.
- Selective 6-bit quantization on AMD MI350X hardware yields 7.6% to 14.3% end-to-end inference throughput gains over FP8 configurations.
Context
Recommendation and Ranking models require massive inference throughput but are highly sensitive to precision loss. Unlike LLMs, production-grade R&R models enforce a strict 0.01% Normalized Entropy margin constraint, making aggressive quantization difficult. This paper addresses this by evaluating MXFP6 and MXFP8 formats on AMD MI350X architecture.
Limitations
The study is limited to two production ads ranking models on AMD MI350X architecture. Results are based on empirical evaluations rather than theoretical predictions. The paper is a preprint (arXiv:2611.17501v1) and does not include peer review or external validation.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 16/11/2026, 05:46:51 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeie7mat2ehnxys47hti6vni4ifb6iqkgwlzpvavsf6qypyb4h3whz4