Multi-Context Fusion Transformer for Pedestrian Crossing Intention Prediction in Urban Environments

TL;DR

This paper introduces a Multi-Context Fusion Transformer that integrates diverse contextual information to significantly improve pedestrian crossing intention prediction accuracy in urban environments for autonomous vehicles.

Contribution

The paper presents a novel multi-context fusion Transformer architecture that effectively combines behavior, environment, localization, and vehicle motion data for better prediction accuracy.

Findings

Achieves 73%, 93%, and 90% accuracy on JAADbeh, JAADall, and PIE datasets.

Outperforms state-of-the-art methods in pedestrian intention prediction.

Extensive ablation studies confirm the effectiveness of the multi-context fusion approach.

Abstract

Pedestrian crossing intention prediction is essential for autonomous vehicles to improve pedestrian safety and reduce traffic accidents. However, accurate pedestrian intention prediction in urban environments remains challenging due to the multitude of factors affecting pedestrian behavior. In this paper, we propose a multi-context fusion Transformer (MFT) that leverages diverse numerical contextual attributes across four key dimensions, encompassing pedestrian behavior context, environmental context, pedestrian localization context and vehicle motion context, to enable accurate pedestrian intention prediction. MFT employs a progressive fusion strategy, where mutual intra-context attention enables reciprocal interactions within each context, thereby facilitating feature sequence fusion and yielding a context token as a context-specific representation. This is followed by mutual cross-context attention, which integrates features across contexts with a global CLS token serving as a compact multi-context representation. Finally, guided intra-context attention refines context tokens within each context through directed interactions, while guided cross-context attention strengthens the global CLS token to promote multi-context fusion via guided information propagation, yielding deeper and more efficient integration. Experimental results validate the superiority of MFT over state-of-the-art methods, achieving accuracy rates of 73%, 93%, and 90% on the JAADbeh, JAADall, and PIE datasets, respectively. Extensive ablation studies are further conducted to investigate the effectiveness of the network architecture and contribution of different input context. Our code is open-source: https://github.com/ZhongHang0307/Multi-Context-Fusion-Transformer.