AI
Learning to Learn from Context: Synthetic Training from Perturbed Public Documents
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 26/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 04:24:08 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 04:18:25 UTC.
Briefings
This study introduces a synthetic training pipeline using perturbed public documents to teach LLMs context-dependent reasoning without human annotation. By rewriting documents, generating reasoning questions, and filtering samples that genuinely depend on the text, the authors trained a student model on CL-bench. The pipeline produced 10k samples from 3.5k documents, boosting performance from 13.7% to 22.8% on SFT and to 24.6% with reward RL, matching a trillion-parameter frontier model. Improvements were observed in long-context understanding and instruction following, though code generation and knowledge retention remained flat.
Key points
- The pipeline rewrites source documents to reduce memorization risk before generating reasoning questions and rubrics.
- Only samples that genuinely depend on the document are admitted, ensuring the model learns context rather than memorization.
Context and limitations
The study uses perturbed public documents rather than human-annotated contexts; code generation and knowledge retention remained flat despite performance gains.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 04:24:08 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigy3uzkjpbmk6g67ymhntvrlnc4qycmuenfnzvpsrsee2hq2nm46i