Curriculum-Based Iterative Self-Play for Scalable Multi-Drone Racing

TL;DR

CRUISE introduces a scalable reinforcement learning framework using curriculum learning and self-play to train autonomous multi-drone racing agents, significantly improving speed, success rate, and scalability in simulation.

Contribution

It presents a novel curriculum-based self-play method that enhances scalability and performance of multi-drone racing policies in high-fidelity simulations.

Findings

CRUISE nearly doubles the racing speed compared to baselines.

It maintains high success rates across increasing agent densities.

Ablation studies highlight the importance of curriculum structure.

Abstract

The coordination of multiple autonomous agents in high-speed, competitive environments represents a significant engineering challenge. This paper presents CRUISE (Curriculum-Based Iterative Self-Play for Scalable Multi-Drone Racing), a reinforcement learning framework designed to solve this challenge in the demanding domain of multi-drone racing. CRUISE overcomes key scalability limitations by synergistically combining a progressive difficulty curriculum with an efficient self-play mechanism to foster robust competitive behaviors. Validated in high-fidelity simulation with realistic quadrotor dynamics, the resulting policies significantly outperform both a standard reinforcement learning baseline and a state-of-the-art game-theoretic planner. CRUISE achieves nearly double the planner's mean racing speed, maintains high success rates, and demonstrates robust scalability as agent density increases. Ablation studies confirm that the curriculum structure is the critical component for this performance leap. By providing a scalable and effective training methodology, CRUISE advances the development of autonomous systems for dynamic, competitive tasks and serves as a blueprint for future real-world deployment.