Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions

TL;DR

This paper presents a practical, low-latency, high-accuracy visual perception system for autonomous racing cars, addressing challenges and providing solutions for real-time perception in safety-critical environments.

Contribution

It introduces adaptations of computer vision algorithms, loss function improvements, and synchronization techniques tailored for high-speed autonomous racing applications.

Findings

Achieved high accuracy in real-world racing scenarios

Demonstrated low-latency perception suitable for safety-critical tasks

Validated system performance through extensive experiments

Abstract

Autonomous racing provides the opportunity to test safety-critical perception pipelines at their limit. This paper describes the practical challenges and solutions to applying state-of-the-art computer vision algorithms to build a low-latency, high-accuracy perception system for DUT18 Driverless (DUT18D), a 4WD electric race car with podium finishes at all Formula Driverless competitions for which it raced. The key components of DUT18D include YOLOv3-based object detection, pose estimation, and time synchronization on its dual stereovision/monovision camera setup. We highlight modifications required to adapt perception CNNs to racing domains, improvements to loss functions used for pose estimation, and methodologies for sub-microsecond camera synchronization among other improvements. We perform a thorough experimental evaluation of the system, demonstrating its accuracy and low-latency in real-world racing scenarios.