AI

AI labs report benchmark scores 6.4% higher than independent testers, on average

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 19/10/2026, 20:47:26 UTC.

Briefings

An open-source AI benchmarking project compared official lab scores against independent tester results, finding AI labs reported scores 6.4% higher on average for the same models and tests. The study analyzed 1,629 published results from 611 comparisons of 180 models across 103 benchmarks, with all data and code released under open licenses.

Key points

• AI labs reported benchmark scores 6.4% higher than independent testers on average for the same models and tests.
• The study analyzed 1,629 published benchmark results from 611 comparisons of 180 models across 103 benchmarks.

Context

This analysis compares official AI lab scores against independent tester results using open-source data. The study analyzed 1,629 published benchmark results from 611 comparisons of 180 models across 103 benchmarks, with all data and code released under open licenses.

Limitations

Evidence covers only 1,629 published benchmark results from 611 comparisons of 180 models across 103 benchmarks. No additional background or context is supplied beyond the source evidence.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 19/10/2026, 21:08:03 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeid55ycfqamvukpnunp7cofpn43jbzuekjw54aiifom362ekq5xfra