AI
AnswerMap: Faithful Spatial Interpretability of VLMs from Answer Posteriors
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 30/09/2026, 02:58:07 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 29/09/2026, 14:18:24 UTC.
Briefings
AnswerMap is a training-free, black-box visual rationale for Vision-Language Models (VLMs) that constructs spatial maps from answer posteriors. By dividing images into KxK bands and computing the outer product of 'yes' relevance posteriors, it generates a query-conditioned spatial map. This approach bypasses text token reliance for continuous outputs, offering a task-agnostic tool validated across four models and three query distributions. Key findings include an AUC of 0.85 for agreement with model points versus 0.38 for attention, and a 53% flip rate for region deletion compared to 19% for attention. The method supports multiple read-outs including maximum, expectation, and top-mass regions to fix model errors.
Key points
- AnswerMap constructs a spatial map by computing the outer product of 'yes' relevance posteriors from a frozen model's answer head, bypassing text token reliance for continuous outputs.
- The method validates faithfulness by comparing generated map regions against model points (AUC 0.85) and testing region deletion (53% flip rate vs 19% for attention).
Context and limitations
AnswerMap requires KxK image banding and fixed read-out definitions (e.g., expectation, maximum) to function; validation is limited to four models and three query distributions with two tests.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 30/09/2026, 02:58:07 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeibfajeir2p4uawhwoshyrvvcbqnhvvo2k5tc4e74jdrm64x2lke7m