Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Control

TL;DR

This paper introduces DMLoco, a diffusion-based multi-task learning framework combined with online reinforcement learning for robust, language-conditioned quadruped robot control, enabling efficient, real-time adaptation and task transition.

Contribution

It presents a novel integration of diffusion models with online RL for multi-task, language-guided quadruped control, addressing stability and data limitations.

Findings

Achieved stable, language-guided locomotion in simulation and real-world tests.

Enabled onboard control at 50Hz with optimized diffusion sampling.

Demonstrated robust task transitions and adaptability in diverse scenarios.

Abstract

Recent research has highlighted the powerful capabilities of imitation learning in robotics. Leveraging generative models, particularly diffusion models, these approaches offer notable advantages such as strong multi-task generalization, effective language conditioning, and high sample efficiency. While their application has been successful in manipulation tasks, their use in legged locomotion remains relatively underexplored, mainly due to compounding errors that affect stability and difficulties in task transition under limited data. Online reinforcement learning (RL) has demonstrated promising results in legged robot control in the past years, providing valuable insights to address these challenges. In this work, we propose DMLoco, a diffusion-based framework for quadruped robots that integrates multi-task pretraining with online PPO finetuning to enable language-conditioned control and robust task transitions. Our approach first pretrains the policy on a diverse multi-task dataset using diffusion models, enabling language-guided execution of various skills. Then, it finetunes the policy in simulation to ensure robustness and stable task transition during real-world deployment. By utilizing Denoising Diffusion Implicit Models (DDIM) for efficient sampling and TensorRT for optimized deployment, our policy runs onboard at 50Hz, offering a scalable and efficient solution for adaptive, language-guided locomotion on resource-constrained robotic platforms.