AI
Beyond Timestamps: Decision-Aligned On-Policy Distillation for Long-Horizon Agents
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 26/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 03:22:23 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 03:18:24 UTC.
Briefings
This article presents AlignOPSD, a method to correct Decision-Timestamp Mismatch in reinforcement learning by aligning supervision before credit assignment. Evaluated on Qwen2.5-3B and Qwen2.5-7B with eight backbone metrics, AlignOPSD outperforms GRPO by 5.5–8.7% and ranks first in six comparisons. The approach uses functionally matched contexts across sibling rollouts to calibrate local teacher evidence and derives variable-duration decision spans from correspondence changes.
Key points
- AlignOPSD corrects Decision-Timestamp Mismatch by aligning supervision before credit assignment, addressing misalignment between privileged guidance and student decisions across different timesteps.
- The method uses functionally matched contexts across sibling rollouts to calibrate local teacher evidence and derives variable-duration decision spans from correspondence changes.
Context and limitations
Evaluation is limited to Qwen2.5-3B and Qwen2.5-7B on ALFWorld, WebShop, and Search-QA; results may vary across other backbones or tasks.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 03:22:23 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeiad2vhdfmldfz55aip35p5vv7iprwpi5guaddqlt7lcfxzaal6zli