AI
SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 30/09/2026, 03:21:23 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 15:18:24 UTC.
Briefings
This article presents SCOPD, a training-free framework for efficient vision-language model pruning that addresses the 'representation-utilization gap' where useful visual information is lost during aggressive compression. By using a privileged teacher to supervise on-policy reasoning trajectories from pruned tokens, SCOPD achieves 92.43% performance on 13 benchmarks at 10% visual-token retention, significantly outperforming the Vanilla model's 86.37%. The study concludes that efficient reasoning depends on both which visual information survives pruning and how reliably the model learns to use it.
Key points
- SCOPD is a training-free framework that uses a privileged teacher to supervise on-policy reasoning trajectories from pruned visual tokens, achieving 92.43% performance on 13 benchmarks at 10% visual-token retention.
- The representation-utilization gap explains why aggressive pruning degrades performance by showing that useful visual evidence can remain accessible but be used unreliably during sampling.
Context and limitations
Results are based on a fixed-context Pass@K analysis and are limited to 13 benchmarks; performance at other token budgets or with different pruning operators remains untested.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 30/09/2026, 03:21:23 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeicunjkgjimo525xkkn6bgonucuj4c3bhymofs7bs4cw72izgfvjk4