A Comprehensive Review on Traffic Datasets and Simulators for Autonomous Vehicles

TL;DR

This review comprehensively analyzes traffic datasets and simulators, emphasizing their roles in advancing autonomous vehicle technology across perception, localization, prediction, planning, and control, and explores emerging trends for system robustness.

Contribution

It uniquely integrates analysis of datasets and simulators across the entire AV pipeline, highlighting their interconnections and recent innovations for improved system resilience.

Findings

Datasets vary by geographic and environmental diversity affecting reliability.

Simulators are specialized for different research domains and contribute to edge case testing.

Emerging trends include multimodal AI and advanced data generation techniques.

Abstract

Autonomous driving has rapidly evolved through synergistic developments in hardware and artificial intelligence. This comprehensive review investigates traffic datasets and simulators as dual pillars supporting autonomous vehicle (AV) development. Unlike prior surveys that examine these resources independently, we present an integrated analysis spanning the entire AV pipeline-perception, localization, prediction, planning, and control. We evaluate annotation practices and quality metrics while examining how geographic diversity and environmental conditions affect system reliability. Our analysis includes detailed characterizations of datasets organized by functional domains and an in-depth examination of traffic simulators categorized by their specialized contributions to research and development. The paper explores emerging trends, including novel architecture frameworks, multimodal AI integration, and advanced data generation techniques that address critical edge cases. By highlighting the interconnections between real-world data collection and simulation environments, this review offers researchers a roadmap for developing more robust and resilient autonomous systems equipped to handle the diverse challenges encountered in real-world driving environments.