Image Generation for Efficient Neural Network Training in Autonomous Drone Racing

TL;DR

This paper introduces a semi-synthetic dataset generation method for training neural networks to improve autonomous drone racing, enabling reliable gate detection and navigation in real-time without extensive real-world data collection.

Contribution

A novel semi-synthetic dataset approach combining real backgrounds and 3D renders to enhance neural network training for drone racing.

Findings

Successful real-time detection and navigation in drone racing scenarios.

Robustness to background and lighting variations.

Efficient training process with synthetic data.

Abstract

Drone racing is a recreational sport in which the goal is to pass through a sequence of gates in a minimum amount of time while avoiding collisions. In autonomous drone racing, one must accomplish this task by flying fully autonomously in an unknown environment by relying only on computer vision methods for detecting the target gates. Due to the challenges such as background objects and varying lighting conditions, traditional object detection algorithms based on colour or geometry tend to fail. Convolutional neural networks offer impressive advances in computer vision but require an immense amount of data to learn. Collecting this data is a tedious process because the drone has to be flown manually, and the data collected can suffer from sensor failures. In this work, a semi-synthetic dataset generation method is proposed, using a combination of real background images and randomised 3D renders of the gates, to provide a limitless amount of training samples that do not suffer from those drawbacks. Using the detection results, a line-of-sight guidance algorithm is used to cross the gates. In several experimental real-time tests, the proposed framework successfully demonstrates fast and reliable detection and navigation.