AI

StoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator Agents

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 09/10/2026, 04:03:24 UTC.

Briefings

StoreBench is a live-commerce environment testing autonomous agents on a mid-size apparel store backend. It evaluates seven frontier LLMs over three world seeds using 11 scenarios spanning 30 to 45 days and a full simulated year. Human experts consistently outperform all models, with DeepSeek-V4-Pro achieving only 49% pass rate against a scripted smart-triage policy. A GRPO post-training run on Qwen3.5-27B increased its mean composite score from 0.136 to 0.373 on held-out tasks. The benchmark uses a windowed operation budget where simulated time depends on actions, preventing latency from influencing outcomes.

Key points

• StoreBench evaluates seven frontier LLMs on 11 scenarios of 30 to 45 days and a full simulated year across three world seeds.
• Human experts consistently outperform all models, with DeepSeek-V4-Pro achieving only 49% pass rate against a scripted smart-triage policy.

Context and limitations

The benchmark uses a windowed operation budget where simulated time depends on actions, preventing latency from influencing outcomes. The full environment and evaluation suite are withheld to keep the benchmark uncontaminated.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 09/10/2026, 04:36:50 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeig2vhfaxvmtnpq25ixv2vqd2zr6gdhpdtkkcpa66mlctptsppkf6q