AI
LoRAWeaver: Directional Adapter Routing in LoRA Representation Space
Publication: cs.AI updates on arXiv.org
Author: Inish Melroy D Souza, Tom Jackson, Vitor Castro, Rebecca Higginson, Andrea Soltoggio
Source published: 16/11/2026, 05:00:00 UTC
Briefing created: 16/11/2026, 05:32:05 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 16/11/2026, 05:03:24 UTC.
Briefings
A new training-free method for adapter routing in LoRA models uses cosine similarity to match adapter-conditioned queries against task prototypes, selecting adapters without retraining. Experiments on LLaMA-2-7B and LLaMA-2-13B with 48 tasks and out-of-distribution tasks show the method outperforms training-free baselines in cross-task settings and approaches a trained retrieval encoder.
Key points
- The proposed method selects adapters by matching adapter-conditioned query representations against task prototypes using cosine similarity.
- Experiments on LLaMA-2-7B and LLaMA-2-13B with 48 tasks and out-of-distribution tasks show the method outperforms training-free baselines in the cross-task setting.
Context
Large language models are expensive to update through full retraining. Parameter-efficient fine-tuning reduces the cost of adaptation, but repeated adaptation creates a growing pool of task-specific LoRA adapters. This pool complicates adapter selection during inference. Existing routing and encoder-based methods often require retraining when new adapters are added, which limits their use in dynamic settings.
Limitations
The study uses preprint arXiv:2611.17340v1, so results may not be fully validated. Experiments were conducted on LLaMA-2-7B and LLaMA-2-13B with 48 tasks and out-of-distribution tasks, limiting generalizability to other architectures or task sets. The method approaches a trained retrieval encoder without learning a router, which may not fully address all routing challenges.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 16/11/2026, 05:32:05 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeifdd7qksrkj4h344bjyuuv7heayrr63ql7gsjs2wot2ylhoa3fj7m