AI

Beyond On-Policy vs. Off-Policy: At What Level Should LLM Agent Distillation Follow the Student?

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 26/10/2026, 04:03:24 UTC.

Briefings

A new hierarchical agent distillation framework formalizes the distinction between teacher-generated demonstrations and student-generated experience levels. The study, conducted via self-distillation across three models and four interactive environments, found that student histories paired with teacher action prefixes yield the strongest standalone training signal, while following the student at both levels performed worst. Mixtures with identical teacher/student proportions at each level showed variable performance depending on pairing, and front-loading teacher contexts improved results over static mixtures.

Key points

• The study introduces a hierarchical agent distillation framework that formalizes two distinct levels of on-policyness: the policy inducing interaction histories versus the policy generating token prefixes.
• Self-distillation experiments across three models and four interactive environments revealed that student-induced histories paired with teacher-generated action prefixes provide the strongest standalone training signal.

Context

This research addresses the challenge of distilling interactive LLM agents by distinguishing between teacher-generated demonstrations and student-generated experience levels. The findings offer practical principles for deciding where, how much, and when agent distillation should follow the student, addressing a critical gap in current distillation methodologies.

Limitations

The study relies on self-distillation methodology and a controlled three-stage study design. Results are limited to the three models and four interactive environments tested, and the specific pairing effects observed may not generalize to all agent distillation scenarios.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 26/10/2026, 06:46:22 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeidlklsaaw6lpgl3t2gnxxnv5wdgh36zxh5oocp6wjglp4yepcztfu