On complementing end-to-end human behavior predictors with planning

TL;DR

This paper explores combining end-to-end human behavior prediction with planning-based methods to improve robustness and accuracy, especially in out-of-distribution scenarios, using simple detection techniques in autonomous driving.

Contribution

It analyzes methods for switching between predictors based on input distribution, highlighting simple classifiers as effective solutions.

Findings

Simple classifiers can effectively detect when to switch predictors.

Ensembling and generative models may not reliably identify out-of-distribution inputs.

Combining approaches improves robustness in autonomous driving predictions.

Abstract

High capacity end-to-end approaches for human motion (behavior) prediction have the ability to represent subtle nuances in human behavior, but struggle with robustness to out of distribution inputs and tail events. Planning-based prediction, on the other hand, can reliably output decent-but-not-great predictions: it is much more stable in the face of distribution shift (as we verify in this work), but it has high inductive bias, missing important aspects that drive human decisions, and ignoring cognitive biases that make human behavior suboptimal. In this work, we analyze one family of approaches that strive to get the best of both worlds: use the end-to-end predictor on common cases, but do not rely on it for tail events / out-of-distribution inputs -- switch to the planning-based predictor there. We contribute an analysis of different approaches for detecting when to make this switch, using an autonomous driving domain. We find that promising approaches based on ensembling or generative modeling of the training distribution might not be reliable, but that there very simple methods which can perform surprisingly well -- including training a classifier to pick up on tell-tale issues in predicted trajectories.