AI

UNREAL: Unifying Retrieval and Long-Context with a Single Model

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 07/10/2026, 08:18:24 UTC.

Briefings

UNREAL is a model-native framework that unifies retrieval and long-context inference by selecting evidence directly from the frozen LLM's internal representations. It achieves state-of-the-art performance on dense and hybrid backbones, improving recall on HotpotQA by 24.1% and on 2WikiMultiHopQA by 28.4%. The system reduces FLOPs and time-to-first-token relative to full-context inference from roughly 32K tokens onward, with larger gains as context grows. This establishes model-internal evidence selection as a common foundation for corpus retrieval and evidence-sparse long-context inference.

Key points

• UNREAL introduces a model-native evidence selection framework that encodes chunks and derives retrieval queries directly from the frozen LLM's internal representations.
• On a 3B-token, 21M-chunk Wikipedia index, all four dense and hybrid UNREAL backbones outperform state-of-the-art retriever-reranker systems, raising recall on HotpotQA from 49.1% to 73.2%.

Context

Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. UNREAL addresses this gap by providing a unified mechanism for evidence selection across these scales.

Limitations

Results are based on a single 3B-token, 21M-chunk Wikipedia index snapshot. Performance gains on HotpotQA and 2WikiMultiHopQA are reported at specific context lengths (128K and 256K tokens respectively). NoLiMa accuracy and LV-Eval F1 scores are reported at maximum context lengths of 128K and 256K tokens, with uncertainty regarding performance at other lengths.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 26/10/2026, 03:31:28 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeifznqpywiwsekprvhbiix44blxrhuys7xvdh4qiicynxom3agbftm