AI

Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 14/10/2026, 12:19:26 UTC.

Briefings

MOSAIC is a new method for attributing influence to training samples in vision-language models, using sparse image-text latent spaces to locate contributions at the patch level. Evaluated on Qwen and Gemma across captioning, chest X-ray understanding, and visual question answering, the approach successfully localizes influence to specific text phrases and image regions without degrading task performance. Ablation studies confirm that influencing specific latent concepts produces more significant output changes than random or activation-ranked latents.

Key points

• MOSAIC introduces a multimodal orthogonalized sparse autoencoder that learns a shared sparse image-text latent space aligned with specific phrases and image regions.
• The method uses EK-FAC to decompose training-example influence and localize contributions to specific text phrases and image regions.

Context

This research addresses the gap in existing attribution methods that lack multimodal and spatial structure for grounding influence in vision-language models, offering a new way to trace training-example influence to multimodal evidence.

Limitations

Evaluation is limited to Qwen and Gemma across three tasks; uncertainty exists regarding generalizability across different model architectures and datasets.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 25/10/2026, 19:07:44 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeiahaw6ouh2licztvijua7bgq5sjdiftl6zbifkbil2nxwzrcwgp5i