AI

AutoEMatic: Runtime-Adaptive Mixed-Precision LLM Inference via Automatic Exponent-Mantissa Refinement

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 10/11/2026, 08:33:27 UTC.

Briefings

AutoEMatic is a runtime-adaptive system that dynamically adjusts LLM inference precision using entropy and update magnitude signals. It achieves up to 21.7% higher accuracy on reasoning benchmarks compared to fixed-precision NVFP4 while maintaining 90-98% decode throughput on GB300 GPUs. The system utilizes AutoFP, a multi-scale format derived from a single NVFP4 base, to optimize both accuracy and latency.

Key points

• AutoEMatic uses two runtime signals—entropy of the next-token distribution and the magnitude of layer updates—to select precision dynamically at every inference step.
• The system improves accuracy by up to 21.7% over NVFP4 on reasoning benchmarks while matching group-scaled FP6 accuracy on most benchmarks.

Context

This research addresses the challenge of varying model precision requirements across prompts and inference phases, proposing a full-stack co-design for runtime adaptation.

Limitations

Performance data is limited to GB300 GPUs; latency and energy savings are reported relative to an NVFP4-only accelerator baseline at iso-accuracy.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 10/11/2026, 10:19:43 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeibc3lpmjdgrtzltokcfwlr7lgwhp6c342fbz5wrvcrc5dgghtapoa