Context-aware Pedestrian Trajectory Prediction with Multimodal Transformer

TL;DR

This paper introduces a multimodal transformer model for pedestrian trajectory prediction that predicts entire future paths in a single pass, outperforming existing methods in accuracy and speed on PIE and JAAD datasets.

Contribution

The novel multimodal transformer architecture predicts full trajectories at once, enabling faster and more accurate pedestrian prediction suitable for edge deployment.

Findings

Achieves lowest error at 0.5, 1.0, and 1.5 seconds horizons

Significantly faster than previous state-of-the-art methods

Ablation studies highlight the importance of multimodal configuration

Abstract

We propose a novel solution for predicting future trajectories of pedestrians. Our method uses a multimodal encoder-decoder transformer architecture, which takes as input both pedestrian locations and ego-vehicle speeds. Notably, our decoder predicts the entire future trajectory in a single-pass and does not perform one-step-ahead prediction, which makes the method effective for embedded edge deployment. We perform detailed experiments and evaluate our method on two popular datasets, PIE and JAAD. Quantitative results demonstrate the superiority of our proposed model over the current state-of-the-art, which consistently achieves the lowest error for 3 time horizons of 0.5, 1.0 and 1.5 seconds. Moreover, the proposed method is significantly faster than the state-of-the-art for the two datasets of PIE and JAAD. Lastly, ablation experiments demonstrate the impact of the key multimodal configuration of our method.