AI
OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 29/09/2026, 20:00:00 UTC
Briefing created: 01/10/2026, 09:00:50 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 01/10/2026, 07:18:25 UTC.
Briefings
OSWorld-Science is a benchmark environment for evaluating visual language models in scientific workflows, featuring 12 VLMs and 146 tasks across domains like molecular drawing and pathology analysis. Tasks are evaluated via artifact inspection, and results indicate current SOTA models still struggle with key scientific questions despite advanced harnesses. The benchmark connects expert goals to verifiable outcomes for systematic agent capability assessment.
Key points
- The benchmark includes 12 visual language models and 146 high-quality tasks across scientific domains such as molecular drawing, retrosynthesis, and pathology image analysis.
- Current state-of-the-art VLMs with integrated harnesses still face challenges in addressing key questions within scientific workflows.
Context and limitations
Evaluation relies on artifact inspection and partial credit for incomplete outcomes; results are limited to the 12 models and 146 tasks tested in this specific benchmark.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 01/10/2026, 09:00:50 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeifluwx6v2dp7qgn7i2mz6ydz5ziqncbioy7v2ivu5vmzbib5ep5hy