AI

Into the Unknown: Benchmarking AI agents on open scientific research problems

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 16/10/2026, 02:33:24 UTC.

Briefings

A new benchmark called Into the Unknown evaluates AI agents on open scientific research problems by providing them with a science paper and a research direction to write a follow-up paper. Nine frontier agents were tested across five domains, producing 15 submissions each. None of the 135 submissions met all conditions, with 114 failing integrity checks due to unverified claims or contradictory data. The study is publicly available.

Key points

• Nine frontier AI agents were evaluated on open scientific research problems using a benchmark called Into the Unknown.
• Agents produced 15 follow-up papers each, totaling 135 submissions, but none met all conditions or passed integrity checks.

Context

This benchmark tests whether AI agents can perform research on open scientific problems by evaluating their ability to write follow-up papers based on provided research directions and papers. The study uses conditions from US Department of Energy scientists to create a rubric for evaluating submissions.

Limitations

The study covers only nine frontier agents across five domains. No submissions met all conditions, and 114 failed integrity checks due to unverified claims or contradictory data. The code and data are publicly available.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 25/10/2026, 17:44:12 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeiemyetwveg75uelvhwctsnorzeu54gxldvpuasbc67qz4vgkjezoa