ProSkill: Segment-Level Skill Assessment in Procedural Videos
Michele Mazzamuto, Daniele Di Mauro, Gianpiero Francesca, Giovanni Maria Farinella, Antonino Furnari

TL;DR
ProSkill introduces a large-scale, action-level skill assessment benchmark dataset for procedural videos, enabling more comprehensive and objective evaluation of human skills in complex tasks.
Contribution
It presents the first dataset with absolute and pairwise skill annotations for procedural tasks, along with a scalable annotation protocol and benchmarking of current algorithms.
Findings
Current state-of-the-art algorithms perform suboptimally on ProSkill.
ProSkill enables evaluation of both ranking-based and pairwise skill assessment methods.
The dataset highlights challenges in automatic skill assessment for complex procedural videos.
Abstract
Skill assessment in procedural videos is crucial for the objective evaluation of human performance in settings such as manufacturing and procedural daily tasks. Current research on skill assessment has predominantly focused on sports and lacks large-scale datasets for complex procedural activities. Existing studies typically involve only a limited number of actions, focus on either pairwise assessments (e.g., A is better than B) or on binary labels (e.g., good execution vs needs improvement). In response to these shortcomings, we introduce ProSkill, the first benchmark dataset for action-level skill assessment in procedural tasks. ProSkill provides absolute skill assessment annotations, along with pairwise ones. This is enabled by a novel and scalable annotation protocol that allows for the creation of an absolute skill assessment ranking starting from pairwise assessments. This…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Sports Performance and Training · Motor Control and Adaptation
