AI
On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 28/09/2026, 20:00:00 UTC
Briefing created: 05/10/2026, 03:25:34 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 05/10/2026, 03:18:24 UTC.
Briefings
This study investigates whether on-policy parameter update directions can improve generalization in supervised fine-tuning (SFT). The research identifies that SFT updates follow consistent directions, whereas on-policy paradigms continuously adjust these directions. By analyzing the cumulative update direction of each parameter, the authors propose On-Policy direction-constrained Supervised Fine-Tuning (OPSFT), which constrains SFT updates to the on-policy direction. Strong performance of OPSFT demonstrates that on-policy generalization advantages can be transferred to SFT. The findings offer practical benefits for combining on-policy generalization with SFT's efficiency and trajectory leverage.
Key points
- SFT updates parameters along consistent directions, while on-policy paradigms continuously adjust these directions during training.
- OPSFT constrains SFT updates to the direction identified by on-policy paradigms, achieving strong generalization performance.
Context and limitations
The study evaluates OPSFT's effectiveness for improving generalization but does not report results on specific model architectures, training data sizes, or comparative performance metrics against other methods.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 05/10/2026, 03:25:34 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeie7tpm23xntncyhpvyhmlqetxutqromltuqsbefltm4mwlqir3a4a