AI
CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 27/09/2026, 20:00:00 UTC
Briefing created: 30/09/2026, 04:01:02 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 30/09/2026, 01:18:24 UTC.
Briefings
CineSubBench introduces a benchmark for evaluating LLMs on long-form narrative understanding using multilingual movie subtitles. The dataset comprises 1,012 films with complete subtitle coverage in six languages, generating 6,072 tracks and 8.13 million timestamped entries. Nine LLMs were tested across nine tasks including narrative reconstruction, genre prediction, and cultural rating systems. Results show plot premises are recovered more reliably than event-complete synopses, while cross-lingual consistency varies significantly across models and languages.
Key points
- CineSubBench contains 1,012 films with complete subtitle coverage in six languages, yielding 6,072 tracks and 8.13 million timestamped subtitle entries.
- Across nine LLMs, plot premises are recovered more reliably than event-complete synopses, while cross-lingual consistency varies substantially across models and languages.
Context and limitations
The dataset covers only nine LLMs and nine tasks; evaluation is limited to subtitle-grounded metrics and does not include scene or event structure reconstruction.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 30/09/2026, 04:01:02 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigk2dhimlohku5p24yaepybsveduo46fk5ylpq4kx2zkubfwsbxc4