A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning

TL;DR

This paper demonstrates that recent machine learning advancements enable a quadruped robot to learn walking in just 20 minutes in real-world environments, outperforming classical controllers on challenging terrains.

Contribution

The study shows that combining modern ML algorithms with a carefully tuned controller allows rapid real-world learning of quadruped locomotion within 20 minutes.

Findings

Robot learns walking gait on various terrains

Achieves rapid learning in 20 minutes

Outperforms classical model-based controllers

Abstract

Deep reinforcement learning is a promising approach to learning policies in uncontrolled environments that do not require domain knowledge. Unfortunately, due to sample inefficiency, deep RL applications have primarily focused on simulated environments. In this work, we demonstrate that the recent advancements in machine learning algorithms and libraries combined with a carefully tuned robot controller lead to learning quadruped locomotion in only 20 minutes in the real world. We evaluate our approach on several indoor and outdoor terrains which are known to be challenging for classical model-based controllers. We observe the robot to be able to learn walking gait consistently on all of these terrains. Finally, we evaluate our design decisions in a simulated environment.