HAIC: Humanoid Agile Object Interaction Control via Dynamics-Aware World Model

TL;DR

HAIC introduces a dynamics-aware world model enabling humanoid robots to interact robustly with diverse, underactuated objects without external state estimation, demonstrating success in complex agile tasks.

Contribution

The paper presents a unified framework with a dynamics predictor and spatial occupancy map that improves humanoid robot interaction with underactuated objects without external state estimation.

Findings

Achieves high success rates in agile tasks like skateboarding and cart pushing.

Proactively compensates for inertial perturbations during object manipulation.

Successfully handles multi-object long-horizon tasks with dynamic predictions.

Abstract

Humanoid robots show promise for complex whole-body tasks in unstructured environments. Although Human-Object Interaction (HOI) has advanced, most methods focus on fully actuated objects rigidly coupled to the robot, ignoring underactuated objects with independent dynamics and non-holonomic constraints. These introduce control challenges from coupling forces and occlusions. We present HAIC, a unified framework for robust interaction across diverse object dynamics without external state estimation. Our key contribution is a dynamics predictor that estimates high-order object states (velocity, acceleration) solely from proprioceptive history. These predictions are projected onto static geometric priors to form a spatially grounded dynamic occupancy map, enabling the policy to infer collision boundaries and contact affordances in blind spots. We use asymmetric fine-tuning, where a world model continuously adapts to the student policy's exploration, ensuring robust state estimation under distribution shifts. Experiments on a humanoid robot show HAIC achieves high success rates in agile tasks (skateboarding, cart pushing/pulling under various loads) by proactively compensating for inertial perturbations, and also masters multi-object long-horizon tasks like carrying a box across varied terrain by predicting the dynamics of multiple objects.