PerturbationDrive: A Framework for Perturbation-Based Testing of ADAS

TL;DR

PerturbationDrive is a comprehensive testing framework designed to evaluate the robustness and generalization of ADAS by applying diverse image perturbations and dynamic scenarios in simulation environments.

Contribution

It introduces a versatile framework that combines multiple perturbation techniques with systematic testing methods for ADAS robustness evaluation.

Findings

Effective in identifying vulnerabilities under various perturbations

Supports both offline and online testing environments

Enables systematic exploration of diverse driving scenarios

Abstract

Advanced driver assistance systems (ADAS) often rely on deep neural networks to interpret driving images and support vehicle control. Although reliable under nominal conditions, these systems remain vulnerable to input variations and out-of-distribution data, which can lead to unsafe behavior. To this aim, this tool paper presents the architecture and functioning of PerturbationDrive, a testing framework to perform robustness and generalization testing of ADAS. The framework features more than 30 image perturbations from the literature that mimic changes in weather, lighting, or sensor quality and extends them with dynamic and attention-based variants. PerturbationDrive supports both offline evaluation on static datasets and online closed-loop testing in different simulators. Additionally, the framework integrates with procedural road generation and search-based testing, enabling systematic exploration of diverse road topologies combined with image perturbations. Together, these features allow PerturbationDrive to evaluate robustness and generalization capabilities of ADAS across varying scenarios, making it a reproducible and extensible framework for systematic system-level testing.