From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Driving

TL;DR

This survey reviews how synthetic data, virtual environments, and simulation technologies are advancing autonomous driving, addressing challenges like data scarcity, safety, and generalization for real-world deployment.

Contribution

It provides a comprehensive taxonomy of datasets, tools, and simulation platforms, and analyzes emerging trends and open challenges in simulation-based autonomous driving research.

Findings

Synthetic data enhances perception and planning models.

Digital twin simulation improves system validation.

Domain adaptation strategies bridge synthetic and real-world data.

Abstract

Autonomous driving technologies have achieved significant advances in recent years, yet their real-world deployment remains constrained by data scarcity, safety requirements, and the need for generalization across diverse environments. In response, synthetic data and virtual environments have emerged as powerful enablers, offering scalable, controllable, and richly annotated scenarios for training and evaluation. This survey presents a comprehensive review of recent developments at the intersection of autonomous driving, simulation technologies, and synthetic datasets. We organize the landscape across three core dimensions: (i) the use of synthetic data for perception and planning, (ii) digital twin-based simulation for system validation, and (iii) domain adaptation strategies bridging synthetic and real-world data. We also highlight the role of vision-language models and simulation realism in enhancing scene understanding and generalization. A detailed taxonomy of datasets, tools, and simulation platforms is provided, alongside an analysis of trends in benchmark design. Finally, we discuss critical challenges and open research directions, including Sim2Real transfer, scalable safety validation, cooperative autonomy, and simulation-driven policy learning, that must be addressed to accelerate the path toward safe, generalizable, and globally deployable autonomous driving systems.