Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation

TL;DR

This paper introduces a unified reinforcement learning-based policy for legged robots that jointly models force and position control without force sensors, improving manipulation capabilities and success rates in contact-rich tasks.

Contribution

It presents the first unified policy for legged robots that co-learns force and position control from simulation, enhancing manipulation versatility without relying on force sensors.

Findings

Achieves approximately 39.5% higher success rates in manipulation tasks.

Enables diverse behaviors like position tracking, force application, and compliant interactions.

Validates robustness across quadrupedal and humanoid robots.

Abstract

Robotic loco-manipulation tasks often involve contact-rich interactions with the environment, requiring the joint modeling of contact force and robot position. However, recent visuomotor policies often focus solely on learning position or force control, overlooking their co-learning. In this work, we propose the first unified policy for legged robots that jointly models force and position control learned without reliance on force sensors. By simulating diverse combinations of position and force commands alongside external disturbance forces, we use reinforcement learning to learn a policy that estimates forces from historical robot states and compensates for them through position and velocity adjustments. This policy enables a wide range of manipulation behaviors under varying force and position inputs, including position tracking, force application, force tracking, and compliant interactions. Furthermore, we demonstrate that the learned policy enhances trajectory-based imitation learning pipelines by incorporating essential contact information through its force estimation module, achieving approximately 39.5% higher success rates across four challenging contact-rich manipulation tasks compared to position-control policies. Extensive experiments on both a quadrupedal manipulator and a humanoid robot validate the versatility and robustness of the proposed policy across diverse scenarios.