AI
AutoGUIWorld: Image Generators as Visual World Models for GUI Agent
Publication: Hugging Face Daily Papers
Author: Not available in collected metadata
Source published: 30/09/2026, 20:00:00 UTC
Briefing created: 02/10/2026, 11:40:36 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 02/10/2026, 02:18:25 UTC.
Briefings
AutoGUIWorld is a data generation framework that synthesizes GUI interaction trajectories using image generators and task planners, eliminating the need to deploy actual software environments. By sampling OS contexts and generating conditioned tasks, the system produced 79,266 spatially annotated step-level samples across four operating systems. Fine-tuning Qwen3.5-35B-A3B on these trajectories improved mean task scores on OSWorld from 33.0% to 40.8% and task success rates on ScienceBoard from 14.0% to 32.2%, demonstrating significant performance gains on real desktop and scientific tasks.
Key points
- AutoGUIWorld synthesizes GUI interaction trajectories by combining image generator visual priors with task knowledge from a planner, eliminating the need to deploy or run actual software environments.
- The framework generated 79,266 spatially annotated step-level training samples across Ubuntu, Windows, macOS, and Chrome, enabling fine-tuning of Qwen3.5-35B-A3B to improve OSWorld task scores from 33.0% to 40.8%.
Context and limitations
Coverage is limited to four operating systems (Ubuntu, Windows, macOS, Chrome) and includes preprint status with uncertainty regarding long-term generalizability beyond the initial fine-tuning results.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 02/10/2026, 11:40:36 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeievr5lbmxxpnif74ygfxma44ki5ityplda72csrv6vnvhzrp3d36u