Self-supervised Pretraining for Integrated Prediction and Planning of Automated Vehicles

TL;DR

This paper introduces Plan-MAE, a pretraining framework using masked autoencoders for integrated prediction and planning in automated vehicles, improving trajectory accuracy and safety.

Contribution

It proposes a novel unified pretraining approach that combines scene understanding with vehicle dynamics for better prediction and planning.

Findings

Outperforms existing methods on planning metrics

Effective in learning spatial and social agent interactions

Serves as a strong pretraining step for motion planning

Abstract

Predicting the future of surrounding agents and accordingly planning a safe, goal-directed trajectory are crucial for automated vehicles. Current methods typically rely on imitation learning to optimize metrics against the ground truth, often overlooking how scene understanding could enable more holistic trajectories. In this paper, we propose Plan-MAE, a unified pretraining framework for prediction and planning that capitalizes on masked autoencoders. Plan-MAE fuses critical contextual understanding via three dedicated tasks: reconstructing masked road networks to learn spatial correlations, agent trajectories to model social interactions, and navigation routes to capture destination intents. To further align vehicle dynamics and safety constraints, we incorporate a local sub-planning task predicting the ego-vehicle's near-term trajectory segment conditioned on earlier segment. This pretrained model is subsequently fine-tuned on downstream tasks to jointly generate the prediction and planning trajectories. Experiments on large-scale datasets demonstrate that Plan-MAE outperforms current methods on the planning metrics by a large margin and can serve as an important pre-training step for learning-based motion planner.