PPT: Pretraining with Pseudo-Labeled Trajectories for Motion Forecasting

TL;DR

PPT is a scalable pretraining framework that leverages automatically generated pseudo-labeled trajectories to improve motion forecasting models, especially in low-data and cross-domain scenarios.

Contribution

Introducing PPT, a novel pretraining method using off-the-shelf trajectories, enhancing robustness and generalization in motion forecasting without manual annotation.

Findings

Achieves strong performance with limited labeled data.

Improves cross-domain generalization.

Enhances robustness in multi-class motion forecasting.

Abstract

Accurately predicting how agents move in dynamic scenes is essential for safe autonomous driving. State-of-the-art motion forecasting models rely on datasets with manually annotated or post-processed trajectories. However, building these datasets is costly, generally manual, hard to scale, and lacks reproducibility. They also introduce domain gaps that limit generalization across environments. We introduce PPT (Pretraining with Pseudo-labeled Trajectories), a simple and scalable pretraining framework that uses unprocessed and diverse trajectories automatically generated from off-the-shelf 3D detectors and tracking. Unlike data annotation pipelines aiming for clean, single-label annotations, PPT is a pretraining framework embracing off-the-shelf trajectories as useful signals for learning robust representations. With optional finetuning on a small amount of labeled data, models pretrained with PPT achieve strong performance across standard benchmarks, particularly in low-data regimes, and in cross-domain, end-to-end, and multi-class settings. PPT is easy to implement and improves generalization in motion forecasting.