AI

The Surprising Effectiveness of Deleting Weights in LLM Post-Training

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 11/11/2026, 00:47:25 UTC.

Briefings

A study by the agentic-learning-ai-lab team reports that deleting fewer than 0.05% of weights in large language models achieves performance comparable to full fine-tuning across three post-training regimes. The method, Deletion Tuning, uses a binary mask on gate projections, resulting in checkpoints hundreds to thousands of times smaller than standard fine-tuning. This approach maintains near-base accuracy during reinforcement learning while other methods lose up to 5.4 points. The research was submitted to r/LocalLLaMA and published in NeurIPS 2026.

Key points

• Deletion Tuning achieves performance comparable to full fine-tuning by removing fewer than 0.05% of weights.
• The method uses a binary mask on gate projections to learn a sparse weight configuration.

Context

The study investigates how minimal weight changes can induce new capabilities in large language models, specifically comparing Deletion Tuning against standard fine-tuning and reinforcement learning methods.

Limitations

Evidence is limited to three LLM post-training regimes; specific performance metrics for other models or regions are not provided. The study is preprint status and does not include results from external validation or real-world deployment data.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 11/11/2026, 07:40:39 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeidrfzj5ma3wynlhtyo5xgfjzng5w3zfphvodlb6nkrdqmiolkiylm