On-Board Pedestrian Trajectory Prediction Using Behavioral Features

TL;DR

This paper introduces BA-PTP, a novel method for pedestrian trajectory prediction from on-board cameras that leverages behavioral features and a modality attention mechanism to improve prediction accuracy.

Contribution

It proposes a new encoding strategy that fuses multiple behavioral features for more accurate pedestrian trajectory prediction from visual data.

Findings

Behavioral features improve prediction accuracy.

The modality attention mechanism effectively combines multiple input streams.

Ablation study highlights the importance of different behavioral features.

Abstract

This paper presents a novel approach to pedestrian trajectory prediction for on-board camera systems, which utilizes behavioral features of pedestrians that can be inferred from visual observations. Our proposed method, called Behavior-Aware Pedestrian Trajectory Prediction (BA-PTP), processes multiple input modalities, i.e. bounding boxes, body and head orientation of pedestrians as well as their pose, with independent encoding streams. The encodings of each stream are fused using a modality attention mechanism, resulting in a final embedding that is used to predict future bounding boxes in the image. In experiments on two datasets for pedestrian behavior prediction, we demonstrate the benefit of using behavioral features for pedestrian trajectory prediction and evaluate the effectiveness of the proposed encoding strategy. Additionally, we investigate the relevance of different behavioral features on the prediction performance based on an ablation study.