AI
Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting
Publication: cs.AI updates on arXiv.org
Author: Loay Mualem, Llu\'is Pastor-P\'erez, Vinh Tong, Andrei Manolache, Tanja Bien, Steffen Staab, Mathias Niepert
Source published: 01/10/2026, 04:00:00 UTC
Briefing created: 01/10/2026, 04:04:29 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 04:03:25 UTC.
Briefings
GoldiMask improves discrete diffusion language models by selecting tokens to reveal as context to maximize a submodular objective, balancing context visibility against prediction target value. This approach outperforms uniform random masking across three backbones and three datasets, boosting accuracy in reasoning and code generation tasks. The method also reduces decoding iterations on GSM8K and MATH-500 while maintaining comparable performance at higher confidence thresholds.
Key points
- GoldiMask selects tokens to reveal as context by approximately maximizing a submodular objective that balances context visibility against prediction target value.
- GoldiMask achieves the highest average accuracy across three backbones and three training datasets, demonstrating gains on both reasoning and code generation tasks.
Context and limitations
Results are limited to three backbones and three training datasets; specific performance metrics for other architectures or datasets are not reported.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 04:04:29 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeidjlykpodhcasije6t6yjwjcgpedaaoio7fsdiohb6j3v4uqsjbia