Efficient Learning of A Unified Policy For Whole-body Manipulation and Locomotion Skills

TL;DR

This paper introduces a reinforcement learning approach that integrates a kinematic model to improve the control of quadruped robots with manipulators, enhancing their locomotion and manipulation capabilities through better exploration and optimization.

Contribution

The paper presents a novel RL framework that incorporates explicit manipulator kinematics, effectively guiding exploration and overcoming local optima in complex quadruped manipulation tasks.

Findings

Successful deployment on DeepRobotics X20 robot with Z1 manipulator

Demonstrated superior performance over existing methods

Improved exploration efficiency and task success rates

Abstract

Equipping quadruped robots with manipulators provides unique loco-manipulation capabilities, enabling diverse practical applications. This integration creates a more complex system that has increased difficulties in modeling and control. Reinforcement learning (RL) offers a promising solution to address these challenges by learning optimal control policies through interaction. Nevertheless, RL methods often struggle with local optima when exploring large solution spaces for motion and manipulation tasks. To overcome these limitations, we propose a novel approach that integrates an explicit kinematic model of the manipulator into the RL framework. This integration provides feedback on the mapping of the body postures to the manipulator's workspace, guiding the RL exploration process and effectively mitigating the local optima issue. Our algorithm has been successfully deployed on a DeepRobotics X20 quadruped robot equipped with a Unitree Z1 manipulator, and extensive experimental results demonstrate the superior performance of this approach.