AI
On the Off-Policy Teacher in On-Policy Distillation
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 28/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 09:15:29 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 01:18:24 UTC.
Briefings
On-policy distillation (OPD) creates an asymmetry where student trajectories are on-policy for the student but off-policy for the teacher, causing teacher continuation performance to degrade as prefixes grow longer. The proposed Student-COnditioned Updates of the Teacher (SCOUT) framework addresses this by adapting the teacher to student-generated prefixes using reinforcement learning with verifiable rewards. Controlled experiments across multiple configurations and reasoning domains demonstrate that SCOUT consistently improves the effectiveness of on-policy distillation.
Key points
- On-policy distillation (OPD) creates an asymmetry where sampled trajectories are on-policy for the student but off-policy for the teacher, causing teacher continuation performance to degrade as prefixes grow longer.
- The Student-COnditioned Updates of the Teacher (SCOUT) framework improves the teacher's ability to continue from student-generated prefixes by adapting the teacher using reinforcement learning with verifiable rewards.
Context and limitations
Results are limited to controlled experiments across multiple teacher-student configurations, model scales, and reasoning domains. The study does not report long-term performance degradation trends beyond the observed prefix length effects, nor does it include data on specific model architectures or reasoning domains beyond the tested configurations.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 09:15:29 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeia3lr3lt7tf3tki42om6wqz3ynlqhreta6j2bgc5hwlgqatsmxig4