AI
Scaling Properties of Same-Family On-Policy Distillation
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 25/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 05:03:08 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 30/09/2026, 16:18:24 UTC.
Briefings
Research on on-policy distillation (OPD) reveals that early training dynamics show a regular useful-transfer regime where held-out accuracy rises linearly with the square root of reverse KL divergence. In every weak-to-strong pair observed, the student's peak gold score exceeds the teacher's, suggesting a compact RL expert can transfer capability to a larger student. Power law analysis indicates peak gold score improves with teacher scale only up to roughly the student's scale, meaning teacher score alone does not define supervision value. The study also examines scaling effects of bootstrapping weak-to-strong OPD and the degree of on-policy supervision.
Key points
- Early on-policy distillation training dynamics exhibit a regular useful-transfer regime where held-out accuracy rises approximately linearly with the square root of token-level reverse KL divergence from the student initialization.
- In every observed weak-to-strong teacher-student pair, the student's peak gold score exceeds the teacher's own, indicating a compact RL expert can transfer capability to a much larger student via OPD.
Context and limitations
Results are based on limited evidence covering only weak-to-strong and strong-to-weak setups across same-base and different-base configurations. Power law scaling laws are estimated rather than measured, and specific parameter counts for teacher and student models are not provided.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 05:03:08 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeif2oqksgitjupgyklebbu77lcgvhhfswrct2jamdtyaf6etwvd2da