SparTa: Sparse Graphical Task Models from a Handful of Demonstrations

TL;DR

SparTa introduces a method for learning long-horizon manipulation tasks by inferring task goals through graphical object relationships, enabling robust execution from few demonstrations in diverse environments.

Contribution

The paper presents a novel approach that captures complete object interactions over entire tasks using graphical models, improving robustness and generalization in robot learning from demonstrations.

Findings

Accurate demonstration segmentation and object matching.

Effective learning from multiple demonstrations.

Successful deployment on real robots.

Abstract

Learning long-horizon manipulation tasks efficiently is a central challenge in robot learning from demonstration. Unlike recent endeavors that focus on directly learning the task in the action domain, we focus on inferring what the robot should achieve in the task, rather than how to do so. To this end, we represent evolving scene states using a series of graphical object relationships. We propose a demonstration segmentation and pooling approach that extracts a series of manipulation graphs and estimates distributions over object states across task phases. In contrast to prior graph-based methods that capture only partial interactions or short temporal windows, our approach captures complete object interactions spanning from the onset of control to the end of the manipulation. To improve robustness when learning from multiple demonstrations, we additionally perform object matching using pre-trained visual features. In extensive experiments, we evaluate our method's demonstration segmentation accuracy and the utility of learning from multiple demonstrations for finding a desired minimal task model. Finally, we deploy the fitted models both in simulation and on a real robot, demonstrating that the resulting task representations support reliable execution across environments.