AI
PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 29/09/2026, 09:19:47 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 05:18:25 UTC.
Briefings
PReCache is a training-free framework for multi-LoRA agents that shares KV cache via two designs: PreLRShared and ReBaseShared. PreLRShared precomputes low-rank caches at the start of shared context processing to avoid redundant reprocessing, while ReBaseShared reconstructs the base cache from adapter-free hidden states to minimize error. PreLRShared achieved up to 3.1x TTFT speedup and 2.3x throughput improvement over baseline inference without KV cache sharing. ReBaseShared preserved accuracy with an average drop of only 1.1 points relative to baseline. The study evaluates performance across multiple models and agent benchmarks.
Key points
- PreLRShared precomputes each agent's low-rank cache when the shared context is first processed, allowing the current agent to use its own LR cache without reprocessing context processed by previous agents.
- ReBaseShared reconstructs the shared base cache from adapter-free hidden states, reducing the remaining error caused by the previous agent's adapted representation.
Context and limitations
Performance evaluations were conducted across multiple models and agent benchmarks; specific results for single-stream inference and concurrent serving are provided but not exhaustive.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 29/09/2026, 09:19:47 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeiakhfjhjd7gbjk4jant6xtycshocpwlc3oihvtyt3pyuvnqooeiuy