Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning

TL;DR

This paper demonstrates that multi-agent reinforcement learning enables autonomous racing quadrotors to outperform humans and improve safety in dynamic multi-agent environments.

Contribution

It introduces a multi-agent RL framework for high-speed quadrotor racing, achieving superior performance and safety compared to single-agent approaches.

Findings

Agents outperform champion-level human pilots in multi-player races.

Collision rates are reduced by 50% compared to single-agent baselines.

Zero-shot generalization to safer human interaction is achieved.

Abstract

Autonomous systems have achieved superhuman performance in isolation or simulation, yet they remain brittle in shared, dynamic real-world spaces. This failure stems from the dominant single-agent paradigm for physical applications, where other actors are ignored or treated as environmental noise, preventing effective coordination. Here we show that multi-agent reinforcement learning provides the essential safety scaffolding required for real-world interaction. Using high-speed quadrotor racing as a high-stakes testbed, we train agents to navigate complex aerodynamic interactions and strategic maneuvering with a variable number of racers. Through league-based self-play, agents evolve sophisticated anticipatory behaviors, including proactive collision avoidance, overtaking, and handling multi-agent physical interactions, including aerodynamic downwash. Our agents outperform a champion-level human pilot in multi-player races at speeds exceeding 22 m/s, while simultaneously reducing collision rates by 50 % compared to state-of-the-art single-agent baselines. Crucially, training with diverse artificial agents enables zero-shot generalization to safer human interaction. These results suggest that the path to robust robotic co-existence lies not in isolated safety constraints, but in the rigorous demands of multi-agent interaction. Multimedia materials are available at: https://rpg.ifi.uzh.ch/marl