AI
SMAT: Simple and Efficient Merge-Aware Training
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 26/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 18:37:40 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 08:18:24 UTC.
Briefings
SMAT (Simple and Efficient Merge-Aware Training) is a method designed to improve the performance of merged models by jointly optimizing expert loss and expected loss. The authors identify three common merging operations—scaling, masking, and perturbation—and introduce SMAT to address these. Experiments on four language and vision-language backbones show SMAT improves mean scores by 1.07 to 2.16 points over the strongest baseline, with less than 2% training-time overhead compared to standard fine-tuning. The study was published in September 2026.
Key points
- SMAT introduces periodic scheduling, kernel fusion, and parameter storage switching to reduce training overhead to less than 2% overhead over standard fine-tuning.
- SMAT improves mean scores by 1.07 to 2.16 points over the strongest baseline across four language and vision-language backbones using five merging methods.
Context and limitations
The study covers four language and vision-language backbones and five merging methods. Results are based on simulated merged parameters generated by sampling scaling coefficients, masks, and additive noise. The evidence does not include results from other merging methods or backbones, and the study was published in September 2026.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 18:37:40 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeicobmkv5e6asjjpzwfzdpo7iejgzx3jih7pkkrlbisey47253bvku