The Wilhelm Tell Dataset of Affordance Demonstrations
Rachel Ringe, Mihai Pomarlan, Nikolaos Tsiogkas, Stefano De Giorgis, Maria Hedblom, Rainer Malaka

TL;DR
This paper introduces a new video dataset capturing human demonstrations of household tasks to improve robot perception of affordances, including first- and third-person views and task metadata.
Contribution
The work provides a novel, multi-view affordance demonstration dataset with metadata, enabling better training of perception systems for household robot tasks.
Findings
Dataset includes 7 hours of diverse human demonstrations.
Includes first- and third-person perspectives and task metadata.
Facilitates studying preparatory maneuvers for tasks.
Abstract
Affordances - i.e. possibilities for action that an environment or objects in it provide - are important for robots operating in human environments to perceive. Existing approaches train such capabilities on annotated static images or shapes. This work presents a novel dataset for affordance learning of common household tasks. Unlike previous approaches, our dataset consists of video sequences demonstrating the tasks from first- and third-person perspectives, along with metadata about the affordances that are manifested in the task, and is aimed towards training perception systems to recognize affordance manifestations. The demonstrations were collected from several participants and in total record about seven hours of human activity. The variety of task performances also allows studying preparatory maneuvers that people may perform for a task, such as how they arrange their task space,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
