TBNAI
MemU-Bench: Benchmarking GUI Agents on Long-Horizon User Interactions
Publication: cs.AI updates on arXiv.org
Author: Yizhou Liu, Zhengxi Lu, Fei Tang, Yiwen Qiu, Yong Du, Tongbo Chen, Bofan Chen, Tao Jiang, Wenhao Xu, Yunfeng Shu, Wenqi Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Source published: 03/11/2026, 05:00:00 UTC
Briefing created: 03/11/2026, 10:40:14 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 03/11/2026, 09:03:25 UTC.
Briefings
MemU-Bench introduces a mobile benchmark for GUI agents using 118 vague tasks derived from 29 long-horizon user trajectories. Unlike standard benchmarks, this dataset requires agents to recover missing context from noisy interaction histories rather than relying on precise instructions. The study reveals that state-of-the-art systems like Gemini-3.1-Pro fail significantly under ambiguity, achieving an SPL of only 16.4%, while AppMem successfully boosts GPT-5.5's SPL to 66.9% by distilling history into structured memory.
Key points
- MemU-Bench introduces 118 vague tasks based on 29 long-horizon user trajectories to evaluate GUI agents' ability to recover missing context.
- AppMem improves GPT-5.5's Success weighted by Path Length (SPL) from 15.7% to 66.9%, outperforming a naive summary baseline by 9.3%.
Context
Existing GUI benchmarks rely on precise, stateless instructions, which do not reflect real-world usage where users omit past operation details. This benchmark addresses that gap by testing agents on ambiguous, history-dependent interactions.
Limitations
The evidence is limited to the 29 long-horizon trajectories and 118 tasks used in the study. Results are based on a preprint (arXiv:2611.00191v1) and do not include final peer review or published validation against official application databases.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 03/11/2026, 10:40:14 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeigdv2c5ejckiwziz5x7jct32cuiuw4rdfkiedjmkjyzzx4xey5wl4