Visual CPG-RL: Learning Central Pattern Generators for Visually-Guided Quadruped Locomotion

TL;DR

This paper introduces a novel framework combining central pattern generators with deep reinforcement learning for visually-guided quadruped locomotion, demonstrating improved robustness, energy efficiency, and sim-to-real transfer in navigation tasks.

Contribution

It integrates CPGs into DRL for quadruped locomotion, exploring interoscillator coupling, memory effects, and sim-to-real transfer, which are novel aspects in this context.

Findings

Explicit interoscillator couplings improve navigation robustness.

Memory-enabled policies enhance energy efficiency and robustness.

The approach achieves successful sim-to-real transfer on Unitree Go1.

Abstract

We present a framework for learning visually-guided quadruped locomotion by integrating exteroceptive sensing and central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework. Through both exteroceptive and proprioceptive sensing, the agent learns to coordinate rhythmic behavior among different oscillators to track velocity commands, while at the same time override these commands to avoid collisions with the environment. We investigate several open robotics and neuroscience questions: 1) What is the role of explicit interoscillator couplings between oscillators, and can such coupling improve sim-to-real transfer for navigation robustness? 2) What are the effects of using a memory-enabled vs. a memory-free policy network with respect to robustness, energy-efficiency, and tracking performance in sim-to-real navigation tasks? 3) How do animals manage to tolerate high sensorimotor delays, yet still produce smooth and robust gaits? To answer these questions, we train our perceptive locomotion policies in simulation and perform sim-to-real transfers to the Unitree Go1 quadruped, where we observe robust navigation in a variety of scenarios. Our results show that the CPG, explicit interoscillator couplings, and memory-enabled policy representations are all beneficial for energy efficiency, robustness to noise and sensory delays of 90 ms, and tracking performance for successful sim-to-real transfer for navigation tasks. Video results can be found at https://youtu.be/wpsbSMzIwgM.