AI
Decoding Looped Transformers Better for (Almost) Free
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 30/09/2026, 20:00:00 UTC
Briefing created: 03/10/2026, 07:45:49 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 02/10/2026, 14:18:24 UTC.
Briefings
Looped Transformers improve efficiency by reusing shared blocks across recurrent loops, which naturally generate aligned prediction pairs. The study introduces LoopCD, a training-free contrastive decoding framework that uses earlier recurrent passes to guide token selection. Experiments on four Transformer families show LoopCD-Logits improves Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%, while LoopCD-Hidden boosts Huginn's HumanEval pass@1 from 22.56% to 31.71%. These gains allow halving recurrent loops without exceeding unguided baselines, reducing forward FLOPs by 22.5% to 48.2%.
Key points
- LoopCD-Logits improves Ouro-2.6B-Thinking's AIME 2024 pass@1 from 61.88% to 73.33%.
- LoopCD-Hidden improves Huginn's HumanEval pass@1 from 22.56% to 31.71%.
Context and limitations
Results are limited to four looped Transformer families; gains enable halving loops but do not guarantee full-depth unguided baseline matching across all architectures.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 03/10/2026, 07:45:49 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeiarfsaaejnlqtpmzplvip7oxb6falv2dsv3bpo27gewt3evnwktqu