Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrations

TL;DR

The paper introduces HuMI, a portable framework for learning humanoid whole-body manipulation skills from robot-free demonstrations, significantly improving data efficiency and success rates in diverse environments.

Contribution

HuMI enables robot-free data collection and hierarchical learning for humanoid manipulation, addressing limitations of prior teleoperation and reinforcement learning methods.

Findings

3x increase in data collection efficiency

70% success rate in unseen environments

Effective across five diverse manipulation tasks

Abstract

Current approaches for humanoid whole-body manipulation, primarily relying on teleoperation or visual sim-to-real reinforcement learning, are hindered by hardware logistics and complex reward engineering. Consequently, demonstrated autonomous skills remain limited and are typically restricted to controlled environments. In this paper, we present the Humanoid Manipulation Interface (HuMI), a portable and efficient framework for learning diverse whole-body manipulation tasks across various environments. HuMI enables robot-free data collection by capturing rich whole-body motion using portable hardware. This data drives a hierarchical learning pipeline that translates human motions into dexterous and feasible humanoid skills. Extensive experiments across five whole-body tasks--including kneeling, squatting, tossing, walking, and bimanual manipulation--demonstrate that HuMI achieves a 3x increase in data collection efficiency compared to teleoperation and attains a 70% success rate in unseen environments.