AI
Scaling Laws for Looped Mixture of Experts
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 08:58:54 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 05:18:24 UTC.
Briefings
This article presents Loop Scaling Laws, the first framework jointly modeling recurrence and sparsity in Looped Mixture-of-Experts models. The laws predict held-out loss more accurately than prior alternatives and recover standard dense and MoE scaling laws as special cases. At trillion-token scale, a looped MoE with derived recurrence matches a ~2x larger non-looped MoE on reasoning benchmarks while enabling test-time scaling through recurrence.
Key points
- Loop Scaling Laws are the first scaling law to jointly model recurrence and sparsity alongside model size and data.
- The laws predict the held-out loss of looped models more accurately than prior alternatives.
Context and limitations
The evidence covers only Looped Mixture-of-Experts models at trillion-token scale; specific parameter counts, training epochs, or benchmark datasets are not provided.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 08:58:54 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeifgldwflivvuux4kx25bpx5k5t3nfzektn2vidasneazrmqq4dwea