Unsupervised Skill Discovery as Exploration for Learning Agile Locomotion

TL;DR

This paper introduces SDAX, an unsupervised skill discovery framework that autonomously enables legged robots to learn diverse, agile locomotion behaviors without extensive reward engineering or demonstrations, and successfully transfers to real hardware.

Contribution

SDAX is a novel unsupervised learning approach that reduces human effort and dynamically manages exploration to teach robots complex agile behaviors.

Findings

Robots learned diverse skills like crawling, climbing, and jumping.

SDAX reduces reliance on reward engineering and demonstrations.

Successful transfer of learned policies to real hardware.

Abstract

Exploration is crucial for enabling legged robots to learn agile locomotion behaviors that can overcome diverse obstacles. However, such exploration is inherently challenging, and we often rely on extensive reward engineering, expert demonstrations, or curriculum learning - all of which limit generalizability. In this work, we propose Skill Discovery as Exploration (SDAX), a novel learning framework that significantly reduces human engineering effort. SDAX leverages unsupervised skill discovery to autonomously acquire a diverse repertoire of skills for overcoming obstacles. To dynamically regulate the level of exploration during training, SDAX employs a bi-level optimization process that autonomously adjusts the degree of exploration. We demonstrate that SDAX enables quadrupedal robots to acquire highly agile behaviors including crawling, climbing, leaping, and executing complex maneuvers such as jumping off vertical walls. Finally, we deploy the learned policy on real hardware, validating its successful transfer to the real world.