aUToLights: A Robust Multi-Camera Traffic Light Detection and Tracking System

TL;DR

This paper presents a multi-camera traffic light detection and tracking system for autonomous vehicles that integrates YOLOv5, semantic map priors, and hidden Markov models to enhance robustness in complex urban scenarios.

Contribution

The authors introduce a multi-camera, real-time traffic light perception system that combines deep learning detection, map priors, and state filtering, demonstrating improved accuracy in challenging conditions.

Findings

Superior performance in occlusion and flashing light scenarios

Effective multi-camera fusion improves detection robustness

Validated on a diverse, annotated dataset

Abstract

Following four successful years in the SAE AutoDrive Challenge Series I, the University of Toronto is participating in the Series II competition to develop a Level 4 autonomous passenger vehicle capable of handling various urban driving scenarios by 2025. Accurate detection of traffic lights and correct identification of their states is essential for safe autonomous operation in cities. Herein, we describe our recently-redesigned traffic light perception system for autonomous vehicles like the University of Toronto's self-driving car, Artemis. Similar to most traffic light perception systems, we rely primarily on camera-based object detectors. We deploy the YOLOv5 detector for bounding box regression and traffic light classification across multiple cameras and fuse the observations. To improve robustness, we incorporate priors from high-definition semantic maps and perform state filtering using hidden Markov models. We demonstrate a multi-camera, real time-capable traffic light perception pipeline that handles complex situations including multiple visible intersections, traffic light variations, temporary occlusion, and flashing light states. To validate our system, we collected and annotated a varied dataset incorporating flashing states and a range of occlusion types. Our results show superior performance in challenging real-world scenarios compared to single-frame, single-camera object detection.