Tech
The most misunderstood graph in AI just got a major update
Publication: MIT Technology Review
Author: Grace Huckins
Source published: 20/11/2026, 18:15:15 UTC
Briefing created: 20/11/2026, 19:20:30 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 20/11/2026, 19:05:24 UTC.
Briefings
A METR report updated in November 2026 reveals AI models are improving exponentially on verifiable coding tasks but lagging on judgment-heavy open-ended tasks. Models now score 'borderline above the bar' on new evaluation design tasks, whereas a year ago they were 'clearly below the bar.' Experts note that while progress on verifiable tasks continues, extrapolation for hard-to-verify tasks remains uncertain due to design and scoring difficulties.
Key points
- METR's updated time horizon plot shows AI models doubling the length of tasks they can complete every four months, indicating exponential improvement on verifiable coding and math tasks.
- Models currently score 'borderline above the bar' on new evaluation design tasks, whereas a year ago they were 'clearly below the bar,' marking a significant shift in performance.
Context
METR evaluated models by scoring task completion length based on human testing times. The report adds new tasks including open-ended research direction selection and philosophical argument scoring, which are harder to verify than coding. Experts note that while progress on verifiable tasks continues, extrapolation for hard-to-verify tasks remains uncertain due to design and scoring difficulties.
Limitations
The report's original suite of tasks was widely misinterpreted due to complexity; the new suite includes more open-ended tasks but METR includes few of them in its report, making it difficult to predict progress on these areas. Experts note that while progress on verifiable tasks continues, extrapolation for hard-to-verify tasks remains uncertain due to design and scoring difficulties. The report does not specify the exact number of new tasks added or the specific dates of the new evaluation design task improvements.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 20/11/2026, 19:20:30 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeid7okwgygmkrdv3z5k5ytbyabedfnhsxcf754ozzbrp5jc7mnys7e