AI

Incongruence-Aware Evaluation of Multimodal Speech Emotion Recognition in Audio Large Language Models

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 02/11/2026, 05:03:24 UTC.

Briefings

A new study investigates whether discrepancies between spoken words and vocal tone degrade emotion recognition in audio large language models. The researchers developed an incongruence score based on the distance between text and audio, validated against human judgment, and evaluated three audiocapable models. Results show that this incongruence score increases error rates when audio is present, whereas supervised classifiers trained on the same features perform better, indicating the issue lies in how models combine evidence rather than the signal itself.

Key points

• The study introduces an incongruence score calculated as the distance between text and audio channels, validated against human judgment.
• Incongruence scores increase error rates in audio large language models when audio is present, while supervised classifiers trained on the same features perform better.

Context

Voice-interactive large language models process both spoken words and vocal tone, yet these channels often disagree. This research addresses whether that disagreement degrades speech emotion recognition.

Limitations

The incongruence score predicts error in all three corpora only when rebuilt without the emotion lexicon. Code, data tables, and prompts are released for reproducibility.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 02/11/2026, 06:08:51 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeiggxt2svqirxa3o733xyewjpxzskr5nurhfq7xv43g5i6s3i3khxa