AI

Video2Skill: From Streaming Experience to Reusable Embodied Skills

Read full article at source ↗

Based on a feed excerpt; full article content has not been acquired. Source evidence retrieved 06/10/2026, 16:18:24 UTC.

Briefings

Video2Skill evaluates whether Vision-Language Models can transform streaming video data into reusable embodied skills. The benchmark tests three capabilities: locating manipulation events, grouping similar transformations, and deciding whether to reuse or create new skills. Across 19 open-source VLMs, most models fail to group events at near-chance levels, and scaling does not consistently improve performance. Key findings indicate that joint models merge distinct transformations into single skills, while text-based updates cause duplication of recurring events. Supervised fine-tuning improves grouping but fails to expand the skill library, resulting in libraries that stall below half the reference size.

Key points

• Video2Skill benchmark tests three capabilities: locating manipulation events, grouping similar transformations, and deciding whether to reuse or create new skills.
• Across 19 open-source VLMs, most models fail to group events at near-chance levels, and scaling does not consistently improve performance.

Context

This research addresses the challenge of transforming streaming video data into reusable embodied skills for future agent training. The benchmark covers robot tabletop manipulation and human kitchen activity, providing a systematic evaluation of how well models can organize observed experience into structured knowledge.

Limitations

Evidence is limited to 19 open-source VLMs, with no additional models or datasets provided. Uncertainty exists regarding whether scaling consistently improves performance or if errors depend on specific coupling between perception and library updates. The study does not report causal claims about why certain models succeed or fail, only descriptive observations of performance gaps.

Generated by Bittrees-hosted qwen35-2b sha256:aaf42c8b7c3c on 22/10/2026, 06:46:43 UTC. Based on publisher-provided feed evidence; may not cover the full original article. Check the source for details.

View archived briefing on IPFS
bafybeiawkmo2qmmwdo5r456d3qgwgrvkixel62sjmguydglkt6xa6p4zca