Embrace Collisions: Humanoid Shadowing for Deployable Contact-Agnostics Motions

TL;DR

This paper introduces a real-time, GPU-accelerated humanoid control framework that enables robots to interact with their environment using all body parts, handling unpredictable contacts and large movements effectively.

Contribution

It presents a novel humanoid motion control framework utilizing GPU simulation and reinforcement learning to manage complex, contact-rich motions in real time.

Findings

Successful real-time control of humanoids with stochastic contacts

Effective handling of large torso rotations and complex motions

Robustness to unpredictable contact sequences

Abstract

Previous humanoid robot research works treat the robot as a bipedal mobile manipulation platform, where only the feet and hands contact the environment. However, we humans use all body parts to interact with the world, e.g., we sit in chairs, get up from the ground, or roll on the floor. Contacting the environment using body parts other than feet and hands brings significant challenges in both model-predictive control and reinforcement learning-based methods. An unpredictable contact sequence makes it almost impossible for model-predictive control to plan ahead in real time. The success of the zero-shot sim-to-real reinforcement learning method for humanoids heavily depends on the acceleration of GPU-based rigid-body physical simulator and simplification of the collision detection. Lacking extreme torso movement of the humanoid research makes all other components non-trivial to design, such as termination conditions, motion commands and reward designs. To address these potential challenges, we propose a general humanoid motion framework that takes discrete motion commands and controls the robot's motor action in real time. Using a GPU-accelerated rigid-body simulator, we train a humanoid whole-body control policy that follows the high-level motion command in the real world in real time, even with stochastic contacts and extremely large robot base rotation and not-so-feasible motion command. More details at https://project-instinct.github.io