# Deep Drone Racing: From Simulation to Reality with Domain Randomization

## TL;DR

This paper presents a modular vision-based drone racing system trained in simulation with domain randomization, enabling zero-shot transfer to real drones for high-speed racing in dynamic environments.

## Contribution

It introduces a novel approach combining CNN perception with planning and control, trained solely in simulation for real-world drone racing without fine-tuning.

## Key findings

- System achieves zero-shot sim-to-real transfer.
- Significant robustness to illumination and appearance changes.
- Outperforms existing state-of-the-art methods.

## Abstract

Dynamically changing environments, unreliable state estimation, and operation under severe resource constraints are fundamental challenges that limit the deployment of small autonomous drones. We address these challenges in the context of autonomous, vision-based drone racing in dynamic environments. A racing drone must traverse a track with possibly moving gates at high speed. We enable this functionality by combining the performance of a state-of-the-art planning and control system with the perceptual awareness of a convolutional neural network (CNN). The resulting modular system is both platform- and domain-independent: it is trained in simulation and deployed on a physical quadrotor without any fine-tuning. The abundance of simulated data, generated via domain randomization, makes our system robust to changes of illumination and gate appearance. To the best of our knowledge, our approach is the first to demonstrate zero-shot sim-to-real transfer on the task of agile drone flight. We extensively test the precision and robustness of our system, both in simulation and on a physical platform, and show significant improvements over the state of the art.