Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation

TL;DR

This paper introduces TRILL, a data-efficient deep imitation learning framework that enables humanoid robots to learn loco-manipulation skills from human demonstrations via VR, addressing challenges of high degrees of freedom and demonstration collection.

Contribution

The paper presents TRILL, a novel framework combining VR-based data collection, whole-body control, and high-level action abstractions for efficient humanoid loco-manipulation learning.

Findings

Effective in simulation and real-world tasks

Reduces data collection effort for humanoid learning

Achieves complex loco-manipulation skills

Abstract

We tackle the problem of developing humanoid loco-manipulation skills with deep imitation learning. The difficulty of collecting task demonstrations and training policies for humanoids with a high degree of freedom presents substantial challenges. We introduce TRILL, a data-efficient framework for training humanoid loco-manipulation policies from human demonstrations. In this framework, we collect human demonstration data through an intuitive Virtual Reality (VR) interface. We employ the whole-body control formulation to transform task-space commands by human operators into the robot's joint-torque actuation while stabilizing its dynamics. By employing high-level action abstractions tailored for humanoid loco-manipulation, our method can efficiently learn complex sensorimotor skills. We demonstrate the effectiveness of TRILL in simulation and on a real-world robot for performing various loco-manipulation tasks. Videos and additional materials can be found on the project page: https://ut-austin-rpl.github.io/TRILL.