Robust Roadside Perception: an Automated Data Synthesis Pipeline Minimizing Human Annotation

TL;DR

This paper introduces an automated data synthesis pipeline using augmented reality and GANs to generate realistic roadside perception data, improving robustness and transferability of perception systems under diverse weather and lighting conditions.

Contribution

It presents a novel method combining AR and GANs to synthesize high-quality training data for roadside perception, reducing reliance on manual annotation and enhancing system robustness.

Findings

Synthesized data improves perception accuracy across various conditions.

Combining synthesized with real data enhances existing detector performance.

The approach is validated at real-world intersections in Michigan.

Abstract

Recently, advancements in vehicle-to-infrastructure communication technologies have elevated the significance of infrastructure-based roadside perception systems for cooperative driving. This paper delves into one of its most pivotal challenges: data insufficiency. The lacking of high-quality labeled roadside sensor data with high diversity leads to low robustness, and low transfer-ability of current roadside perception systems. In this paper, a novel solution is proposed to address this problem that creates synthesized training data using Augmented Reality. A Generative Adversarial Network is then applied to enhance the reality further, that produces a photo-realistic synthesized dataset that is capable of training or fine-tuning a roadside perception detector which is robust to different weather and lighting conditions. Our approach was rigorously tested at two key intersections in Michigan, USA: the Mcity intersection and the State St./Ellsworth Rd roundabout. The Mcity intersection is located within the Mcity test field, a controlled testing environment. In contrast, the State St./Ellsworth Rd intersection is a bustling roundabout notorious for its high traffic flow and a significant number of accidents annually. Experimental results demonstrate that detectors trained solely on synthesized data exhibit commendable performance across all conditions. Furthermore, when integrated with labeled data, the synthesized data can notably bolster the performance of pre-existing detectors, especially in adverse conditions.