Active Exploration for Robotic Manipulation

TL;DR

This paper introduces a model-based active exploration method for robotic manipulation that uses ensemble models and MPC to efficiently learn contact-rich tasks, demonstrated on a real robot pushing a ball on tilted tables.

Contribution

It presents a novel active exploration approach combining ensemble probabilistic models and MPC for efficient learning in complex manipulation tasks.

Findings

Effective in simulation and real robot experiments

Enables learning from scratch with sparse rewards

Successfully manipulates objects with unknown target positions

Abstract

Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in recent years. One of the key challenges in manipulation is the exploration of the dynamics of the environment when there is continuous contact between the objects being manipulated. This paper proposes a model-based active exploration approach that enables efficient learning in sparse-reward robotic manipulation tasks. The proposed method estimates an information gain objective using an ensemble of probabilistic models and deploys model predictive control (MPC) to plan actions online that maximize the expected reward while also performing directed exploration. We evaluate our proposed algorithm in simulation and on a real robot, trained from scratch with our method, on a challenging ball pushing task on tilted tables, where the target ball position is not known to the agent a-priori. Our real-world robot experiment serves as a fundamental application of active exploration in model-based reinforcement learning of complex robotic manipulation tasks.