MultiPark: Multimodal Parking Transformer with Next-Segment Prediction

TL;DR

MultiPark introduces a multimodal transformer model with next-segment prediction and outcome-oriented loss to improve parking maneuver accuracy and safety in complex, lane-free environments, demonstrating state-of-the-art results and real-world robustness.

Contribution

It presents MultiPark, a novel autoregressive transformer with a next-segment prediction paradigm and parking queries, addressing causal confusion and multimodal behavior in parking scenarios.

Findings

Achieves state-of-the-art performance on real-world datasets.

Demonstrates robustness in real-world vehicle deployment.

Effectively models diverse parking behaviors.

Abstract

Parking accurately and safely in highly constrained spaces remains a critical challenge. Unlike structured driving environments, parking requires executing complex maneuvers such as frequent gear shifts and steering saturation. Recent attempts to employ imitation learning (IL) for parking have achieved promising results. However, existing works ignore the multimodal nature of parking behavior in lane-free open space, failing to derive multiple plausible solutions under the same situation. Notably, IL-based methods encompass inherent causal confusion, so enabling a neural network to generalize across diverse parking scenarios is particularly difficult. To address these challenges, we propose MultiPark, an autoregressive transformer for multimodal parking. To handle paths filled with abrupt turning points, we introduce a data-efficient next-segment prediction paradigm, enabling spatial generalization and temporal extrapolation. Furthermore, we design learnable parking queries factorized into gear, longitudinal, and lateral components, parallelly decoding diverse parking behaviors. To mitigate causal confusion in IL, our method employs target-centric pose and ego-centric collision as outcome-oriented loss across all modalities beyond pure imitation loss. Evaluations on real-world datasets demonstrate that MultiPark achieves state-of-the-art performance across various scenarios. We deploy MultiPark on a production vehicle, further confirming our approach's robustness in real-world parking environments.