Efficient Bimanual Manipulation Using Learned Task Schemas

TL;DR

This paper introduces a method for bimanual robotic manipulation that decomposes skill learning into a task schema and parameterization policies, significantly improving sample efficiency and transferability in sparse-reward tasks.

Contribution

It proposes explicitly modeling task schemas as state-independent sequences of skills, enhancing learning efficiency and enabling transfer to related tasks in robotic manipulation.

Findings

Improved sample efficiency in learning robotic skills.

Successful transfer of schemas to related tasks.

Effective real-world robotic manipulation demonstrated.

Abstract

We address the problem of effectively composing skills to solve sparse-reward tasks in the real world. Given a set of parameterized skills (such as exerting a force or doing a top grasp at a location), our goal is to learn policies that invoke these skills to efficiently solve such tasks. Our insight is that for many tasks, the learning process can be decomposed into learning a state-independent task schema (a sequence of skills to execute) and a policy to choose the parameterizations of the skills in a state-dependent manner. For such tasks, we show that explicitly modeling the schema's state-independence can yield significant improvements in sample efficiency for model-free reinforcement learning algorithms. Furthermore, these schemas can be transferred to solve related tasks, by simply re-learning the parameterizations with which the skills are invoked. We find that doing so enables learning to solve sparse-reward tasks on real-world robotic systems very efficiently. We validate our approach experimentally over a suite of robotic bimanual manipulation tasks, both in simulation and on real hardware. See videos at http://tinyurl.com/chitnis-schema.