AI
KV-LTM: Persistent Per-Session KV Caches as Long-Term Memory for Personal Assistants
Publication: cs.AI updates on arXiv.org
Author: Ziv Kfir, Ido Kfir
Source published: 18/11/2026, 05:00:00 UTC
Briefing created: 18/11/2026, 05:58:01 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 18/11/2026, 05:03:25 UTC.
Briefings
This briefing reports on a proposed KV-LTM architecture for personal assistants, which stores raw session text alongside KV caches under a fixed reader model to restore cached sessions and append new questions. The system is designed to be complementary to retrieval systems, trading storage for reduced re-prefill compute and write-time fidelity, with a retention budget derived from the model's attention mechanism. Evaluation protocols are defined to test logical QA without persistent cache implementation, with a separate minimal real-cache replay used to measure numerical equivalence and compute costs.
Key points
- KV-LTM proposes storing each session's raw text plus its KV cache under a fixed reader model to restore cached sessions and append the question.
- Per-token state spans approximately 31K-328K bytes for pinned BF16 full-attention (GQA) and latent-cached MLA configurations, resulting in approximately 0.34-3.6 TB per user-year at a 30K-token/day scenario.
Context
Personal assistants accumulate months of conversations, but current long-term memory stacks often struggle with derivation or selection between a question and its evidence. This proposal addresses content loss between write and read operations by using KV-LTM as a memory tier for prefill acceleration.
Limitations
Evaluation protocols are defined to test logical QA without persistent cache implementation, with a separate minimal real-cache replay used to measure numerical equivalence and compute costs. The system is complementary to, not a replacement for, retrieval and indexing systems.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 18/11/2026, 05:58:01 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeidch27bu5kylhfrnmiw3bwxklq3a6ex6365h6rc5nwllsapuaxfte