Test Automation for Interactive Scenarios via Promptable Traffic Simulation

TL;DR

This paper presents an automated approach to generate realistic and safety-critical human behaviors in traffic simulations for evaluating autonomous vehicle planners, using promptable simulation and Bayesian optimization.

Contribution

It introduces a novel method combining promptable traffic simulation with Bayesian optimization to automate the creation of diverse, realistic, and safety-critical test scenarios for AV evaluation.

Findings

Successfully generates diverse safety-critical scenarios

Efficiently explores goal space with Bayesian optimization

Demonstrates effectiveness on an optimization-based AV planner

Abstract

Autonomous vehicle (AV) planners must undergo rigorous evaluation before widespread deployment on public roads, particularly to assess their robustness against the uncertainty of human behaviors. While recent advancements in data-driven scenario generation enable the simulation of realistic human behaviors in interactive settings, leveraging these models to construct comprehensive tests for AV planners remains an open challenge. In this work, we introduce an automated method to efficiently generate realistic and safety-critical human behaviors for AV planner evaluation in interactive scenarios. We parameterize complex human behaviors using low-dimensional goal positions, which are then fed into a promptable traffic simulator, ProSim, to guide the behaviors of simulated agents. To automate test generation, we introduce a prompt generation module that explores the goal domain and efficiently identifies safety-critical behaviors using Bayesian optimization. We apply our method to the evaluation of an optimization-based planner and demonstrate its effectiveness and efficiency in automatically generating diverse and realistic driving behaviors across scenarios with varying initial conditions.