SimADFuzz: Simulation-Feedback Fuzz Testing for Autonomous Driving Systems

TL;DR

SimADFuzz is a novel simulation-feedback fuzz testing framework that improves scenario generation for autonomous driving systems by leveraging violation prediction and distance-guided mutations, uncovering more safety violations.

Contribution

It introduces violation prediction models and distance-guided mutation strategies to generate diverse, effective testing scenarios for ADS, surpassing existing fuzzers in violation detection.

Findings

Identified 32 more unique violations than state-of-the-art methods.

Discovered 4 reproducible collision cases involving vehicles and pedestrians.

Enhanced the robustness and safety testing of autonomous driving systems.

Abstract

Autonomous driving systems (ADS) have achieved remarkable progress in recent years. However, ensuring their safety and reliability remains a critical challenge due to the complexity and uncertainty of driving scenarios. In this paper, we focus on simulation testing for ADS, where generating diverse and effective testing scenarios is a central task. Existing fuzz testing methods face limitations, such as overlooking the temporal and spatial dynamics of scenarios and failing to leverage simulation feedback (e.g., speed, acceleration and heading) to guide scenario selection and mutation. To address these issues, we propose SimADFuzz, a novel framework designed to generate high-quality scenarios that reveal violations in ADS behavior. Specifically, SimADFuzz employs violation prediction models, which evaluate the likelihood of ADS violations, to optimize scenario selection. Moreover, SimADFuzz proposes distance-guided mutation strategies to enhance interactions among vehicles in offspring scenarios, thereby triggering more edge-case behaviors of vehicles. Comprehensive experiments demonstrate that SimADFuzz outperforms state-of-the-art fuzzers by identifying 32 more unique violations, including 4 reproducible cases of vehicle-vehicle and vehicle-pedestrian collisions. These results demonstrate SimADFuzz's effectiveness in enhancing the robustness and safety of autonomous driving systems.