AI
LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 14:11:02 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 13:18:23 UTC.
Briefings
LEAP is a framework for hour-scale audio-visual question answering that decouples evidence localization from reasoning by dividing recordings into fixed-duration blocks. It uses lightweight localization passes to score candidate windows before pooling and re-encoding them in a single bounded answer pass. This approach allows the model to localize temporal windows over pre-computed transcripts without decoding media frames, while preserving fine-grained visual and non-speech evidence. LEAP improves Qwen3-Omni-30B-A3B by 4.5-16.8% and MiniCPM-o 4.5 by 3.1-13.0% across AVQA benchmarks.
Key points
- LEAP divides audio-visual recordings into fixed-duration blocks to localize evidence without encoding the entire recording.
- The framework uses a localization LoRA to score candidate windows and an answer LoRA to improve final answers from selected blocks.
Context and limitations
LEAP supports streaming inference via causal queries but requires pre-computed transcripts for localization. Results are limited to AVQA benchmarks and may vary across different audio-visual modalities.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 14:11:02 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeihkegquc54rrtijyxhx5dnymgrp73ohv4p3ioacml53hfsuupkxou