Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation

TL;DR

This paper introduces a novel unsupervised skill discovery method for robotic manipulation, enabling robots to autonomously learn diverse, composable behaviors that improve interaction robustness and facilitate solving unseen tasks.

Contribution

It proposes a new approach combining autonomous task generation, asymmetric self-play, and multiplicative compositional policies for skill discovery in robotics.

Findings

Skills are more interactive than baselines.

Skills enable solving unseen manipulation tasks.

Method works in both simulation and real robots.

Abstract

Learning skills that interact with objects is of major importance for robotic manipulation. These skills can indeed serve as an efficient prior for solving various manipulation tasks. We propose a novel Skill Learning approach that discovers composable behaviors by solving a large and diverse number of autonomously generated tasks. Our method learns skills allowing the robot to consistently and robustly interact with objects in its environment. The discovered behaviors are embedded in primitives which can be composed with Hierarchical Reinforcement Learning to solve unseen manipulation tasks. In particular, we leverage Asymmetric Self-Play to discover behaviors and Multiplicative Compositional Policies to embed them. We compare our method to Skill Learning baselines and find that our skills are more interactive. Furthermore, the learned skills can be used to solve a set of unseen manipulation tasks, in simulation as well as on a real robotic platform.