A Framework for Deploying Learning-based Quadruped Loco-Manipulation

TL;DR

This paper introduces an open, unified framework for training, benchmarking, and deploying reinforcement learning controllers on quadruped robots with manipulators, bridging simulation and real-world deployment effectively.

Contribution

It presents a reproducible pipeline that unifies sim-to-sim and sim-to-real transfer for quadruped loco-manipulation using RL, with open-source tools and hardware abstraction layers.

Findings

Discrepancies between Isaac Gym and MuJoCo contact models affect policy behavior.

Real-world trials demonstrate improved reach and manipulation with whole-body control.

The framework enables transparent analysis and development of RL-based loco-manipulation controllers.

Abstract

Quadruped mobile manipulators offer strong potential for agile loco-manipulation but remain difficult to control and transfer reliably from simulation to reality. Reinforcement learning (RL) shows promise for whole-body control, yet most frameworks are proprietary and hard to reproduce on real hardware. We present an open pipeline for training, benchmarking, and deploying RL-based controllers on the Unitree B1 quadruped with a Z1 arm. The framework unifies sim-to-sim and sim-to-real transfer through ROS, re-implementing a policy trained in Isaac Gym, extending it to MuJoCo via a hardware abstraction layer, and deploying the same controller on physical hardware. Sim-to-sim experiments expose discrepancies between Isaac Gym and MuJoCo contact models that influence policy behavior, while real-world teleoperated object-picking trials show that coordinated whole-body control extends reach and improves manipulation over floating-base baselines. The pipeline provides a transparent, reproducible foundation for developing and analyzing RL-based loco-manipulation controllers and will be released open source to support future research.