Beyond In-Distribution Performance: A Cross-Dataset Study of Trajectory Prediction Robustness

TL;DR

This study evaluates the out-of-distribution robustness of state-of-the-art trajectory prediction models across datasets, revealing that models with higher inductive bias perform better OoD, but all models struggle when trained on larger datasets and tested on smaller ones.

Contribution

It provides a cross-dataset analysis of trajectory prediction models' OoD generalization, highlighting the impact of model design, data size, and augmentation strategies.

Findings

Smaller, highly biased models generalize better OoD on smaller datasets.

All models perform poorly when trained on larger datasets and tested on smaller ones.

Inductive bias influences OoD robustness more than model size or augmentation.

Abstract

We study the Out-of-Distribution (OoD) generalization ability of three SotA trajectory prediction models with comparable In-Distribution (ID) performance but different model designs. We investigate the influence of inductive bias, size of training data and data augmentation strategy by training the models on Argoverse 2 (A2) and testing on Waymo Open Motion (WO) and vice versa. We find that the smallest model with highest inductive bias exhibits the best OoD generalization across different augmentation strategies when trained on the smaller A2 dataset and tested on the large WO dataset. In the converse setting, training all models on the larger WO dataset and testing on the smaller A2 dataset, we find that all models generalize poorly, even though the model with the highest inductive bias still exhibits the best generalization ability. We discuss possible reasons for this surprising finding and draw conclusions about the design and test of trajectory prediction models and benchmarks.