AdaptManip: Learning Adaptive Whole-Body Object Lifting and Delivery with Online Recurrent State Estimation

TL;DR

AdaptManip is a reinforcement learning framework enabling humanoid robots to autonomously perform integrated navigation, object lifting, and delivery with robust state estimation, outperforming imitation learning methods in real-world tasks.

Contribution

The paper introduces a novel reinforcement learning-based framework for adaptive whole-body loco-manipulation without human demonstrations, incorporating real-time object state estimation and drift-robust localization.

Findings

Outperforms baseline imitation learning methods in success rate and adaptability.

Accurate object state estimation enhances manipulation under occlusion.

Successfully demonstrates autonomous navigation, lifting, and delivery on real humanoid robots.

Abstract

This paper presents Adaptive Whole-body Loco-Manipulation, AdaptManip, a fully autonomous framework for humanoid robots to perform integrated navigation, object lifting, and delivery. Unlike prior imitation learning-based approaches that rely on human demonstrations and are often brittle to disturbances, AdaptManip aims to train a robust loco-manipulation policy via reinforcement learning without human demonstrations or teleoperation data. The proposed framework consists of three coupled components: (1) a recurrent object state estimator that tracks the manipulated object in real time under limited field-of-view and occlusions; (2) a whole-body base policy for robust locomotion with residual manipulation control for stable object lifting and delivery; and (3) a LiDAR-based robot global position estimator that provides drift-robust localization. All components are trained in simulation using reinforcement learning and deployed on real hardware in a zero-shot manner. Experimental results show that AdaptManip significantly outperforms baseline methods, including imitation learning-based approaches, in adaptability and overall success rate, while accurate object state estimation improves manipulation performance even under occlusion. We further demonstrate fully autonomous real-world navigation, object lifting, and delivery on a humanoid robot.