Learning Quadruped Locomotion Policies using Logical Rules

TL;DR

This paper introduces a novel method for learning quadruped gaits using logical rules and reward machines, enabling easy gait specification, efficient policy learning, and real-world robot deployment without extensive manual effort.

Contribution

The paper presents RM-based Locomotion Learning (RMLL), a new approach that uses logical rules and reward machines for high-level gait specification and efficient policy learning in quadruped robots.

Findings

Learned diverse gaits including two novel ones

Achieved superior sample efficiency compared to baselines

Demonstrated stable gaits across different terrains and real robot deployment

Abstract

Quadruped animals are capable of exhibiting a diverse range of locomotion gaits. While progress has been made in demonstrating such gaits on robots, current methods rely on motion priors, dynamics models, or other forms of extensive manual efforts. People can use natural language to describe dance moves. Could one use a formal language to specify quadruped gaits? To this end, we aim to enable easy gait specification and efficient policy learning. Leveraging Reward Machines~(RMs) for high-level gait specification over foot contacts, our approach is called RM-based Locomotion Learning~(RMLL), and supports adjusting gait frequency at execution time. Gait specification is enabled through the use of a few logical rules per gait (e.g., alternate between moving front feet and back feet) and does not require labor-intensive motion priors. Experimental results in simulation highlight the diversity of learned gaits (including two novel gaits), their energy consumption and stability across different terrains, and the superior sample-efficiency when compared to baselines. We also demonstrate these learned policies with a real quadruped robot. Video and supplementary materials: https://sites.google.com/view/rm-locomotion-learning/home