PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction

TL;DR

PhysHOI is a novel physics-based approach that enables humanoid robots to imitate dynamic human-object interactions without task-specific rewards, using contact graphs and a new dataset for basketball skills.

Contribution

This work introduces PhysHOI, the first physics-based whole-body HOI imitation method that eliminates the need for task-specific reward design and models contact relations explicitly.

Findings

PhysHOI successfully imitates diverse HOI tasks including basketball skills.

The contact graph reward is critical for precise imitation.

The BallPlay dataset provides new benchmarks for dynamic HOI imitation.

Abstract

Humans interact with objects all the time. Enabling a humanoid to learn human-object interaction (HOI) is a key step for future smart animation and intelligent robotics systems. However, recent progress in physics-based HOI requires carefully designed task-specific rewards, making the system unscalable and labor-intensive. This work focuses on dynamic HOI imitation: teaching humanoid dynamic interaction skills through imitating kinematic HOI demonstrations. It is quite challenging because of the complexity of the interaction between body parts and objects and the lack of dynamic HOI data. To handle the above issues, we present PhysHOI, the first physics-based whole-body HOI imitation approach without task-specific reward designs. Except for the kinematic HOI representations of humans and objects, we introduce the contact graph to model the contact relations between body parts and objects explicitly. A contact graph reward is also designed, which proved to be critical for precise HOI imitation. Based on the key designs, PhysHOI can imitate diverse HOI tasks simply yet effectively without prior knowledge. To make up for the lack of dynamic HOI scenarios in this area, we introduce the BallPlay dataset that contains eight whole-body basketball skills. We validate PhysHOI on diverse HOI tasks, including whole-body grasping and basketball skills.