AI

Low-Rank Hierarchical Merging for Efficient Long-to-Short Reasoning

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 04/11/2026, 05:18:25 UTC.

Briefings

A new training-free framework called Low-Rank Hierarchical Merging (LHM) combines long chain-of-thought reasoning with concise base model styles by using second-order curvature information instead of linear statistics. Experiments on six mathematical benchmarks with 1.5B and 7B models show LHM achieves over 4% accuracy gain and reduces output length by up to 93.1% compared to state-of-the-art methods like ACM-TIES and ORCA.

Key points

• LHM is a training-free framework using low-rank Hessian approximation and Layer Second-Order Sensitivity (LSOS) index to estimate per-layer fusion difficulty.
• LHM outperforms ACM-TIES and ORCA on six mathematical benchmarks, achieving over 4% accuracy gain and up to 93.1% reduction in output length.

Context

This research addresses the failure of linear interpolation and first-order activation statistics in combining long chain-of-thought models with base models, proposing a new second-order curvature-based approach for efficient long-to-short reasoning.

Limitations

Results are limited to six mathematical benchmarks on 1.5B and 7B models; no additional background context is supplied beyond the abstract.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 04/11/2026, 05:59:41 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeib7szz44lrfhx3rmjclepbcxzu45jycn35x3vbyayvhmylbbgrq4a