AI
One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 07/10/2026, 20:00:00 UTC
Briefing created: 16/11/2026, 14:14:01 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 09/10/2026, 12:18:24 UTC.
Briefings
This research introduces reViT, a recurrent Vision Transformer architecture where a single Transformer block is applied repeatedly to restore depth-specific transformations. By using a normalized-depth coordinate to program a mixture of experts, the model achieves comparable accuracy to full-depth encoders while using only one FFN parameter per depth, significantly reducing storage requirements. The study evaluated reViT on ImageNet-1k and via distillation from DINOv2, identifying weight-space merging as the most effective MoE strategy for this architecture.
Key points
- reViT uses a single Transformer block applied recurrently to restore depth-specific transformations.
- The architecture uses a normalized-depth coordinate to program a mixture of experts, achieving one-FFN budget accuracy.
Context
The work presents a recurrent Vision Transformer (reViT) that restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. This design allows for resampleable trajectories through FFN parameter space, enabling efficient depth prediction with minimal storage.
Limitations
The study evaluates reViT on ImageNet-1k and via distillation from DINOv2, but the specific hardware constraints and inference latency for real-time deployment are not detailed. The paper does not provide a full-scale deployment case study or a comparison against other recurrent architectures in terms of hardware efficiency.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 16/11/2026, 14:14:01 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeibq5m7s3r53vnr7rqivqwnwf7ppues6vbfhxd4rojc7cgj5ehnody