PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer

TL;DR

PlanTRansformer (PTR) is a unified Transformer framework that integrates goal-conditioned prediction, dynamic feasibility, and interaction awareness to improve trajectory prediction and planning in autonomous driving, bridging the gap between these components.

Contribution

The paper introduces PTR, a novel unified Transformer-based model that combines prediction and planning with goal conditioning and interaction modeling, addressing their disconnection.

Findings

Achieves 4.3%/3.5% improvement in mAP over baseline models.

Reduces planning error by 15.5% at 5s horizon.

Demonstrates architecture-agnostic design adaptable to various Transformer models.

Abstract

Trajectory prediction and planning are fundamental yet disconnected components in autonomous driving. Prediction models forecast surrounding agent motion under unknown intentions, producing multimodal distributions, while planning assumes known ego objectives and generates deterministic trajectories. This mismatch creates a critical bottleneck: prediction lacks supervision for agent intentions, while planning requires this information. Existing prediction models, despite strong benchmarking performance, often remain disconnected from planning constraints such as collision avoidance and dynamic feasibility. We introduce Plan TRansformer (PTR), a unified Gaussian Mixture Transformer framework integrating goal-conditioned prediction, dynamic feasibility, interaction awareness, and lane-level topology reasoning. A teacher-student training strategy progressively masks surrounding agent commands during training to align with inference conditions where agent intentions are unavailable. PTR achieves 4.3%/3.5% improvement in marginal/joint mAP compared to the baseline Motion Transformer (MTR) and 15.5% planning error reduction at 5s horizon compared to GameFormer. The architecture-agnostic design enables application to diverse Transformer-based prediction models. Project Website: https://github.com/SelzerConst/PlanTRansformer