Are AI-Generated Driving Videos Ready for Autonomous Driving? A Diagnostic Evaluation Framework

TL;DR

This paper evaluates the reliability of AI-generated driving videos for autonomous driving training and testing, introducing a diagnostic framework, a benchmark, and an evaluator to improve their utility and safety.

Contribution

It presents a systematic diagnostic framework, a new benchmark dataset, and an evaluation method to assess and enhance the use of AI-generated driving videos in autonomous driving.

Findings

Raw AIGVs can impair perception models.

Filtering AIGVs with ADGVE improves perception performance.

Properly filtered AIGVs can complement real-world data effectively.

Abstract

Recent text-to-video models have enabled the generation of high-resolution driving scenes from natural language prompts. These AI-generated driving videos (AIGVs) offer a low-cost, scalable alternative to real or simulator data for autonomous driving (AD). But a key question remains: can such videos reliably support training and evaluation of AD models? We present a diagnostic framework that systematically studies this question. First, we introduce a taxonomy of frequent AIGV failure modes, including visual artifacts, physically implausible motion, and violations of traffic semantics, and demonstrate their negative impact on object detection, tracking, and instance segmentation. To support this analysis, we build ADGV-Bench, a driving-focused benchmark with human quality annotations and dense labels for multiple perception tasks. We then propose ADGVE, a driving-aware evaluator that combines static semantics, temporal cues, lane obedience signals, and Vision-Language Model(VLM)-guided reasoning into a single quality score for each clip. Experiments show that blindly adding raw AIGVs can degrade perception performance, while filtering them with ADGVE consistently improves both general video quality assessment metrics and downstream AD models, and turns AIGVs into a beneficial complement to real-world data. Our study highlights both the risks and the promise of AIGVs, and provides practical tools for safely leveraging large-scale video generation in future AD pipelines.