Learning Agile Locomotion via Adversarial Training

TL;DR

This paper introduces an adversarial multi-agent learning framework for legged robots, enabling the development of agile locomotion controllers with reduced environment design effort through ensemble adversaries.

Contribution

It presents a novel multi-agent adversarial training approach with ensemble adversaries to improve robot agility and reduce manual environment design.

Findings

Adversarial training enhances robot agility significantly.

Ensemble of adversaries is crucial for mastering diverse escaping strategies.

Learned controllers outperform baseline methods in agility tests.

Abstract

Developing controllers for agile locomotion is a long-standing challenge for legged robots. Reinforcement learning (RL) and Evolution Strategy (ES) hold the promise of automating the design process of such controllers. However, dedicated and careful human effort is required to design training environments to promote agility. In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effectively alleviates the laborious environment design effort. In contrast to prior works that used only one adversary, we find that training an ensemble of adversaries, each of which specializes in a different escaping strategy, is essential for the protagonist to master agility. Through extensive experiments, we show that the locomotion controller learned with adversarial training significantly outperforms carefully designed baselines.