AI
Training LLM Judges from Language Feedback via Position-Selective Self-Distillation
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 02/10/2026, 10:17:02 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 19:18:24 UTC.
Briefings
This study investigates training Large Language Models (LLMs) as judges using natural language feedback, specifically addressing subjective tasks where verdicts depend on criterion selection and weighting. The research identifies two distinct signal regimes in Self-Distillation: 'context sharpening,' which concentrates probability on specific feedback-aligned criteria, and 'context spreading,' which distributes probability across alternatives. By introducing position masking based on entropy shifts, the authors retain lower-entropy positions to improve out-of-distribution generalization. Results show self-distilled judges outperform outcome-supervised RL by 2-9 percentage points on subjective subcategories while maintaining competitiveness on objective tasks.
Key points
- The study introduces position masking based on entropy shifts to retain lower-entropy positions in Self-Distillation, improving out-of-distribution generalization over naive Self-Distillation.
- Self-distilled judges outperform outcome-supervised RL by 2-9 percentage points on subjective subcategories while remaining competitive on objective tasks.
Context and limitations
The findings are limited to subjective subcategories and objective tasks; the study does not specify the exact dataset used, nor does it provide details on the specific LLM architectures or training configurations employed.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 02/10/2026, 10:17:02 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeieydfmsm5l4j7pn5zauhirqsv5e672fw2unmmozb6nedu7igx6xfe