Testing the Safety of Self-driving Vehicles by Simulating Perception and Prediction

TL;DR

This paper introduces a scalable simulation method for testing self-driving vehicle safety by directly modeling perception and prediction outputs, avoiding costly sensor simulation and enabling realistic motion planning evaluation.

Contribution

It presents a novel approach that predicts perception and prediction outputs from high-level inputs, improving scalability and realism in self-driving vehicle safety testing.

Findings

Realistic motion planning testing achieved on large-scale datasets

Method reduces cost and complexity of simulation

Enables rapid scenario sketching by test engineers

Abstract

We present a novel method for testing the safety of self-driving vehicles in simulation. We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps. Instead, we directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing. Specifically, we use paired data in the form of ground truth labels and real perception and prediction outputs to train a model that predicts what the online system will produce. Importantly, the inputs to our system consists of high definition maps, bounding boxes, and trajectories, which can be easily sketched by a test engineer in a matter of minutes. This makes our approach a much more scalable solution. Quantitative results on two large-scale datasets demonstrate that we can realistically test motion planning using our simulations.