AI

Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 07/10/2026, 05:18:24 UTC.

Briefings

This study investigates whether expanding alignment coverage during On-Policy Distillation improves learning for mathematical reasoning and code generation. Researchers found that restricting reverse KL to a student-selected top-16 subset of the shared vocabulary at each strict position achieves accuracy comparable to full shared-vocabulary OPD, outperforming cross-tokenizer baselines. The findings suggest prioritizing supervision reliability over maximizing alignment coverage, as compact supervision at strict positions is more effective than broader coverage.

Key points

• On-Policy Distillation trains a student on its own generations using teacher feedback.
• Strict 1:1 groups cover most student-generated tokens despite substantial vocabulary mismatch.

Context

The research examines whether expanding alignment coverage improves learning across three heterogeneous teacher-student pairs on mathematical reasoning and code generation tasks.

Limitations

Accuracy drop from adding span supervision is not fully explained; diagnostics show weak or negative directional agreement with strict gradients at checkpoints from training with only the strict loss.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 25/10/2026, 04:17:59 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeidsfkfsuxtv7l7ijkdix4simqdjf254naypthyzoc26cdyz4dgx5y