RMP-YOLO: A Robust Motion Predictor for Partially Observable Scenarios even if You Only Look Once

TL;DR

RMP-YOLO is a novel motion prediction framework that reconstructs incomplete historical trajectories to improve accuracy in partially observable scenarios, demonstrating robustness and state-of-the-art results.

Contribution

The paper introduces a new paradigm focusing on reconstructing trajectories before prediction, with a scene tokenization and recovery module that enhances robustness to missing data.

Findings

Achieves state-of-the-art performance in the 2024 Waymo Motion Prediction Competition.

Effectively handles missing data and observation noise.

Seamlessly integrates with existing prediction models.

Abstract

We introduce RMP-YOLO, a unified framework designed to provide robust motion predictions even with incomplete input data. Our key insight stems from the observation that complete and reliable historical trajectory data plays a pivotal role in ensuring accurate motion prediction. Therefore, we propose a new paradigm that prioritizes the reconstruction of intact historical trajectories before feeding them into the prediction modules. Our approach introduces a novel scene tokenization module to enhance the extraction and fusion of spatial and temporal features. Following this, our proposed recovery module reconstructs agents' incomplete historical trajectories by leveraging local map topology and interactions with nearby agents. The reconstructed, clean historical data is then integrated into the downstream prediction modules. Our framework is able to effectively handle missing data of varying lengths and remains robust against observation noise, while maintaining high prediction accuracy. Furthermore, our recovery module is compatible with existing prediction models, ensuring seamless integration. Extensive experiments validate the effectiveness of our approach, and deployment in real-world autonomous vehicles confirms its practical utility. In the 2024 Waymo Motion Prediction Competition, our method, RMP-YOLO, achieves state-of-the-art performance, securing third place.