AI
Understanding On-Policy Distillation: A Mechanistic Interpretability Perspective via Sparse Crosscoders
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 30/09/2026, 07:25:57 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 30/09/2026, 07:18:24 UTC.
Briefings
This study investigates on-policy distillation (OPD) in large language models using sparse crosscoders and a new swap readout method. Researchers found that OPD does not create new features but instead reweights existing shared features, with warm-up training significantly altering feature usage patterns before distillation. The findings indicate that students learn from the teacher's feature selection rather than acquiring new knowledge, as demonstrated by the ability to replicate warm-up effects without weight changes.
Key points
- Sparse crosscoders measure feature usage changes across student and teacher checkpoints, revealing that OPD reweights existing features rather than creating new ones.
- Warm-up training on the teacher's rollouts reweights features for conversation format, reasoning style, and mathematical notation, persisting through OPD and improving student accuracy.
Context and limitations
The study covers three OPD settings but only examined SFT warm-up on teacher rollouts; feature firing rates for over 98% of student features remained stable at 20% change. Results are based on preprint data with uncertainty regarding generalizability across different distillation configurations.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 30/09/2026, 07:25:57 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigz3ihbxdpmud476yytlhgcao5hzjr6v5agyrp3c3p7d6e6tfkhu4