Learning Descriptor of Constrained Task from Demonstration

TL;DR

This paper introduces a framework that enables robots to learn and generalize constrained object interactions, like opening books, from demonstrations by creating a comprehensive descriptor including object, grasping, and constraint models.

Contribution

The authors propose a novel descriptor that integrates object, grasping, and constraint information to improve robot learning and generalization for constrained tasks from demonstrations.

Findings

Robot can learn constrained motion models from demonstration.

The framework generalizes to novel objects with different sizes and appearances.

Successful application to book opening tasks.

Abstract

Constrained objects, such as doors and drawers are often complex and share a similar structure in the human environment. A robot needs to interact accurately with constrained objects to safely and successfully complete a task. Learning from Demonstration offers an appropriate path to learn the object structure of the demonstration for unknown objects for unknown tasks. There is work that extracts the kinematic model from motion. However, the gap remains when the robot faces a new object with a similar model but different contexts, e.g. size, appearance, etc. In this paper, we propose a framework that integrates all the information needed to learn a constrained motion from a depth camera into a descriptor of the constrained task. The descriptor consists of object information, grasping point model, constrained model, and reference frame model. By associating constrained learning and reference frame with the constrained object, we demonstrate that the robot can learn the book opening model and parameter of the constraints from demonstration and generalize to novel books.