Versatile Loco-Manipulation through Flexible Interlimb Coordination

TL;DR

This paper introduces ReLIC, a reinforcement learning-based approach enabling robots to flexibly coordinate limbs for diverse loco-manipulation tasks, adapting dynamically between locomotion and manipulation in unstructured environments.

Contribution

The work presents a novel adaptive controller that seamlessly integrates manipulation and locomotion, allowing versatile and robust interlimb coordination for complex tasks.

Findings

Achieved an average success rate of 78.9% on real-world tasks.

Demonstrated versatility across 12 diverse loco-manipulation tasks.

Enabled interface with various task specifications like trajectories and natural language.

Abstract

The ability to flexibly leverage limbs for loco-manipulation is essential for enabling autonomous robots to operate in unstructured environments. Yet, prior work on loco-manipulation is often constrained to specific tasks or predetermined limb configurations. In this work, we present Reinforcement Learning for Interlimb Coordination (ReLIC), an approach that enables versatile loco-manipulation through flexible interlimb coordination. The key to our approach is an adaptive controller that seamlessly bridges the execution of manipulation motions and the generation of stable gaits based on task demands. Through the interplay between two controller modules, ReLIC dynamically assigns each limb for manipulation or locomotion and robustly coordinates them to achieve the task success. Using efficient reinforcement learning in simulation, ReLIC learns to perform stable gaits in accordance with the manipulation goals in the real world. To solve diverse and complex tasks, we further propose to interface the learned controller with different types of task specifications, including target trajectories, contact points, and natural language instructions. Evaluated on 12 real-world tasks that require diverse and complex coordination patterns, ReLIC demonstrates its versatility and robustness by achieving a success rate of 78.9% on average. Videos and code can be found at https://relic-locoman.rai-inst.com.