Local and Global Contextual Features Fusion for Pedestrian Intention Prediction

TL;DR

This paper proposes a multi-modal feature fusion approach combining local pedestrian cues and global scene context to improve pedestrian crossing intention prediction for autonomous vehicles, demonstrating superior results on the JAAD dataset.

Contribution

It introduces a novel fusion of spatio-temporal visual features and scene parsing for enhanced pedestrian intention prediction in autonomous driving scenarios.

Findings

Achieved higher AUC and F1-score than state-of-the-art methods.

Effectively combines local pedestrian behavior with global environmental context.

Demonstrates robustness on the JAAD dataset.

Abstract

Autonomous vehicles (AVs) are becoming an indispensable part of future transportation. However, safety challenges and lack of reliability limit their real-world deployment. Towards boosting the appearance of AVs on the roads, the interaction of AVs with pedestrians including "prediction of the pedestrian crossing intention" deserves extensive research. This is a highly challenging task as involves multiple non-linear parameters. In this direction, we extract and analyse spatio-temporal visual features of both pedestrian and traffic contexts. The pedestrian features include body pose and local context features that represent the pedestrian's behaviour. Additionally, to understand the global context, we utilise location, motion, and environmental information using scene parsing technology that represents the pedestrian's surroundings, and may affect the pedestrian's intention. Finally, these multi-modality features are intelligently fused for effective intention prediction learning. The experimental results of the proposed model on the JAAD dataset show a superior result on the combined AUC and F1-score compared to the state-of-the-art.