AI
SkillBuffBench: Measuring What Agent Skills Contribute across Harnesses, Models, Skill Access Modes and Memory
Publication: cs.AI updates on arXiv.org
Author: Gengyue Dong, Ye Wei, Zheng Zhu, Yidong Han, Jiale Miao, Xiaojun Jia, Guanghui Li
Source published: 18/11/2026, 05:00:00 UTC
Briefing created: 18/11/2026, 05:56:05 UTC
Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 18/11/2026, 05:03:25 UTC.
Briefings
SkillBuffBench is a new benchmark evaluating how agent skills impact performance across harnesses, models, and memory states. It tests skills in materials science and biology using domain expert tasks, comparing three skill access levels (no skill, curated skill, automatic invocation) with four harnesses and multiple models. Results are reported with paired bootstrap uncertainty and cost metrics, with full results and analysis released after the evaluation phase completes.
Key points
- SkillBuffBench evaluates agent skills across four harnesses, four models, and three skill access levels (no skill, curated skill, automatic invocation) with memory states (off and on).
- The benchmark uses domain expert tasks in materials science and biology, with results reported alongside cost, agent rounds, and wall-clock time rather than folded into a single score.
Context
This benchmark addresses a gap in existing research where skill effectiveness is measured only in controlled settings with hand-picked skills, ignoring real-world deployment scenarios where agents choose from installed libraries and carry memory from prior work.
Limitations
Evaluation results are released only after the full run completes; current tasks and skills are limited to materials science and biology, with the suite extending to other domains on a rolling basis. Results include paired, task-level bootstrap uncertainty but do not include cost, agent rounds, or wall-clock time in the final score.
Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 18/11/2026, 05:56:05 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.
View archived briefing on IPFS
bafybeiaptb4e7gi3qustnt7zsuw6y2e2k7fuwmipglzmf2zmnoxckmzmmq