AI
Not Every Token Is Worth Distilling: Selective Supervision for Direct-OPD
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 25/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 10:05:38 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 03:18:25 UTC.
Briefings
Researchers introduced Selective Supervision for Direct-OPD (S^2D-OPD), a method that masks Direct-OPD supervision at low-divergence states to improve accuracy. Using exact constructions, they proved that standard Direct-OPD reward updates can remain unchanged while Jensen-Shannon divergence and KL divergence vanish, even when probability mass vanishes. The method ranks student-sampled states by teacher-reference JSD and retains only the top 10% of states per response. Experiments on AIME and HMMT benchmarks with two teacher pairs and four student models (1.7B to 8B parameters) showed S^2D-OPD improved held-out accuracy over dense Direct-OPD in seven of eight settings, matching it in the eighth without extra forward passes.
Key points
- Selective Supervision for Direct-OPD (S^2D-OPD) masks Direct-OPD supervision at low-divergence states to improve accuracy.
- Exact constructions show that Direct-OPD reward updates can remain unchanged while Jensen-Shannon divergence and KL divergence vanish even when probability mass vanishes.
Context and limitations
Results are limited to two teacher pairs and four student models across AIME and HMMT benchmarks; performance matches dense Direct-OPD in only one of eight settings.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 10:05:38 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeig24ce4k5pdv7auxrtcm5hq3i2224zlh6ebspwclugogvlelkutxe