Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrations

TL;DR

This paper presents a GPU-accelerated simulation framework combined with imitation learning to efficiently train humanoid robots for complex, contact-rich manipulation tasks in unstructured environments.

Contribution

It introduces a novel framework integrating GPU simulation and imitation learning for effective reinforcement learning in dexterous manipulation.

Findings

Robust, natural manipulation behaviors achieved

Efficient RL training with GPU simulation and imitation learning

Interactive VR teleoperation interface developed

Abstract

Dexterous manipulation with anthropomorphic robot hands remains a challenging problem in robotics because of the high-dimensional state and action spaces and complex contacts. Nevertheless, skillful closed-loop manipulation is required to enable humanoid robots to operate in unstructured real-world environments. Reinforcement learning (RL) has traditionally imposed enormous interaction data requirements for optimizing such complex control problems. We introduce a new framework that leverages recent advances in GPU-based simulation along with the strength of imitation learning in guiding policy search towards promising behaviors to make RL training feasible in these domains. To this end, we present an immersive virtual reality teleoperation interface designed for interactive human-like manipulation on contact rich tasks and a suite of manipulation environments inspired by tasks of daily living. Finally, we demonstrate the complementary strengths of massively parallel RL and imitation learning, yielding robust and natural behaviors. Videos of trained policies, our source code, and the collected demonstration datasets are available at https://maltemosbach.github.io/interactive_ human_like_manipulation/.