SynAD: Enhancing Real-World End-to-End Autonomous Driving Models through Synthetic Data Integration

TL;DR

SynAD is a novel framework that combines synthetic data with real-world data to improve the safety and robustness of end-to-end autonomous driving models, addressing the challenge of scenario diversity.

Contribution

It introduces a new method for integrating synthetic scenarios into real-world E2E AD training, including a Map-to-BEV network and a specialized training strategy.

Findings

Enhanced safety performance in autonomous driving models

Effective integration of synthetic and real data demonstrated

Improved scenario diversity for training models

Abstract

Recent advancements in deep learning and the availability of high-quality real-world driving datasets have propelled end-to-end autonomous driving. Despite this progress, relying solely on real-world data limits the variety of driving scenarios for training. Synthetic scenario generation has emerged as a promising solution to enrich the diversity of training data; however, its application within E2E AD models remains largely unexplored. This is primarily due to the absence of a designated ego vehicle and the associated sensor inputs, such as camera or LiDAR, typically provided in real-world scenarios. To address this gap, we introduce SynAD, the first framework designed to enhance real-world E2E AD models using synthetic data. Our method designates the agent with the most comprehensive driving information as the ego vehicle in a multi-agent synthetic scenario. We further project path-level scenarios onto maps and employ a newly developed Map-to-BEV Network to derive bird's-eye-view features without relying on sensor inputs. Finally, we devise a training strategy that effectively integrates these map-based synthetic data with real driving data. Experimental results demonstrate that SynAD effectively integrates all components and notably enhances safety performance. By bridging synthetic scenario generation and E2E AD, SynAD paves the way for more comprehensive and robust autonomous driving models.