AI
Post-Training Leaves Behavioral Shadows on Unrelated Decisions
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 23/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 13:19:30 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 06:18:25 UTC.
Briefings
Research reveals language models can transfer capabilities via unrelated text using a new method called Active Taskless Distillation (ATD). This technique transfers skills using only a single word from the teacher per prompt, probing the 'behavioral shadow' of post-training. In a primary coding experiment with Qwen2.5-1.5B, 5,664nses achieved a 5.34 pp gain on HumanEval+ compared to a control group. The study also observed transfer in scientific knowledge and reading comprehension across various model sizes.
Key points
- Active Taskless Distillation (ATD) transfers capabilities using only a single word from the teacher per prompt, bypassing standard task-specific data.
- The method probes the behavioral shadow of post-training by selecting prompts where the teacher and student's shared ancestor is nearly indifferent between two ordinary words.
Context and limitations
The study covers only the primary coding experiment with Qwen2.5-1.5B. Results on scientific knowledge, commonsense reasoning, and reading comprehension were observed across additional model generations, sizes, and families. The research is presented as a preprint and includes uncertainty regarding the generalizability of findings.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 13:19:30 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigq3a5sjnutjqdm3or46sq5uphgz7vrdxvm3mgfqi3mw7ugxtqewm