VOFA: Visual Object Goal Pushing with Force-Adaptive Control for Humanoids

TL;DR

VOFA is a hierarchical humanoid robot system that robustly pushes objects with unknown properties to goal positions using onboard perception and force-adaptive control, achieving high success rates in simulation and real-world tests.

Contribution

The paper introduces VOFA, a novel visual goal-conditioned humanoid pushing system with a two-level architecture combining visuomotor policy and force-adaptive control, capable of handling unknown object properties.

Findings

Over 90% success in simulation

Over 80% success in real-world trials

Pushes objects up to 17kg, exceeding half of robot's weight

Abstract

The ability to push large objects in a goal-directed manner using onboard egocentric perception is an essential skill for humanoid robots to perform complex tasks such as material handling in warehouses. To robustly manipulate heavy objects to arbitrary goal configurations, the robot must cope with unknown object mass and ground friction, noisy onboard perception, and actuation errors; all in a real-time feedback loop. Existing solutions either rely on privileged object-state information without onboard perception or lack robustness to variations in goal configurations and object physical properties. In this work, we present VOFA, a visual goal-conditioned humanoid loco-manipulation system capable of pushing objects with unknown physical properties to arbitrary goal positions. VOFA consists of a two-level hierarchical architecture with a high-level visuomotor policy and a low-level force-adaptive whole-body controller. The high-level policy processes noisy onboard observations and generates goal-conditioned commands to operate in closed loop across diverse object-goal configurations, while the low-level whole-body controller provides robustness to variations in object physical properties. VOFA is extensively evaluated in both simulation and real-world experiments on the Booster T1 humanoid robot. Our results demonstrate strong performance, achieving over 90% success in simulation and over 80% success in real-world trials. Moreover, VOFA successfully pushes objects weighing up to 17kg, exceeding half of the Booster T1's body weight.