Generalizable Humanoid Manipulation with 3D Diffusion Policies

TL;DR

This paper presents a system enabling humanoid robots to autonomously perform manipulation tasks across diverse real-world scenarios by integrating teleoperation data collection, a 25-DoF humanoid platform, and an improved 3D diffusion policy learning algorithm.

Contribution

The work introduces a novel integrated system combining data collection, a humanoid platform, and a 3D diffusion policy for generalizable manipulation skills.

Findings

Robust manipulation policies learned from a single scene data.

Successful autonomous performance in diverse real-world scenarios.

Over 2000 policy rollouts evaluated on the real robot.

Abstract

Humanoid robots capable of autonomous operation in diverse environments have long been a goal for roboticists. However, autonomous manipulation by humanoid robots has largely been restricted to one specific scene, primarily due to the difficulty of acquiring generalizable skills and the expensiveness of in-the-wild humanoid robot data. In this work, we build a real-world robotic system to address this challenging problem. Our system is mainly an integration of 1) a whole-upper-body robotic teleoperation system to acquire human-like robot data, 2) a 25-DoF humanoid robot platform with a height-adjustable cart and a 3D LiDAR sensor, and 3) an improved 3D Diffusion Policy learning algorithm for humanoid robots to learn from noisy human data. We run more than 2000 episodes of policy rollouts on the real robot for rigorous policy evaluation. Empowered by this system, we show that using only data collected in one single scene and with only onboard computing, a full-sized humanoid robot can autonomously perform skills in diverse real-world scenarios. Videos are available at https://humanoid-manipulation.github.io .