AI

On-Policy Distillation Teaches New Skills but Not New Knowledge

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 09/10/2026, 05:18:24 UTC.

Briefings

Research shows on-policy distillation transfers compositional reasoning skills but not new factual knowledge. Experiments across four models confirm reverse-KL distillation improves multi-step reasoning without expanding parametric memory, while forward KL restores factual transfer. Results apply to recent factual QA and competition math benchmarks.

Key points

• Reverse-KL on-policy distillation reliably transfers compositional skill across unseen reasoning structures while transferring minimal factual knowledge.
• Decoupling the distillation recipe reveals that replacing reverse KL with forward KL restores factual knowledge transfer.

Context

This study investigates whether on-policy distillation expands a model's parametric knowledge or merely teaches it to organize existing knowledge. The findings clarify the mechanism behind reasoning gains in language models.

Limitations

Results are limited to four models from three families and recent factual QA and competition math benchmarks. Uncertainty exists regarding the generalizability of compositional skill transfer across unseen reasoning structures.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 14/11/2026, 13:17:22 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeianjfsolol3px27ypue333t2auay7bitcmsuhstmxhvrg6eid3bpi