Foundation Models for Rapid Autonomy Validation

TL;DR

This paper introduces a behavior foundation model using a masked autoencoder to improve the efficiency and coverage of autonomous vehicle safety validation by prioritizing difficult scenarios and grouping similar ones.

Contribution

It presents a novel application of a masked autoencoder as a behavior foundation model for scenario grouping and difficulty scoring in autonomous vehicle validation.

Findings

Enhanced scenario coverage with prioritized hard scenarios

Faster estimation of collision rates and severity

Effective grouping of similar driving scenarios

Abstract

We are motivated by the problem of autonomous vehicle performance validation. A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter, including rare events, to provide a strong case for safety and show there is no edge-case pathological behavior. Autonomous vehicle companies rely on potentially millions of miles driven in realistic simulation to expose the driving stack to enough miles to estimate rates and severity of collisions. To address scalability and coverage, we propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios. We leverage the foundation model in two complementary ways: we (i) use the learned embedding space to group qualitatively similar scenarios together and (ii) fine-tune the model to label scenario difficulty based on the likelihood of a collision upon simulation. We use the difficulty scoring as importance weighting for the groups of scenarios. The result is an approach which can more rapidly estimate the rates and severity of collisions by prioritizing hard scenarios while ensuring exposure to every kind of driving scenario.