Realistic Extreme Behavior Generation for Improved AV Testing

TL;DR

This paper presents a framework for generating realistic, diverse collision scenarios for autonomous vehicle testing by perturbing predicted trajectories, enabling better diagnosis of AV system robustness.

Contribution

It introduces a data-driven approach to create plausible adversarial collision scenarios grounded in realistic vehicle behavior for AV testing.

Findings

Generated scenarios reveal interpretable failure modes in AV policies.

Clustering of synthetic scenarios identifies representative collision types.

Framework effectively diagnoses AV system weaknesses.

Abstract

This work introduces a framework to diagnose the strengths and shortcomings of Autonomous Vehicle (AV) collision avoidance technology with synthetic yet realistic potential collision scenarios adapted from real-world, collision-free data. Our framework generates counterfactual collisions with diverse crash properties, e.g., crash angle and velocity, between an adversary and a target vehicle by adding perturbations to the adversary's predicted trajectory from a learned AV behavior model. Our main contribution is to ground these adversarial perturbations in realistic behavior as defined through the lens of data-alignment in the behavior model's parameter space. Then, we cluster these synthetic counterfactuals to identify plausible and representative collision scenarios to form the basis of a test suite for downstream AV system evaluation. We demonstrate our framework using two state-of-the-art behavior prediction models as sources of realistic adversarial perturbations, and show that our scenario clustering evokes interpretable failure modes from a baseline AV policy under evaluation.