PIP-Net: Pedestrian Intention Prediction in the Wild

TL;DR

PIP-Net is a new deep learning framework that predicts pedestrian crossing intentions in real-world urban environments, utilizing multi-camera data and novel features to improve accuracy and forecast up to 4 seconds ahead.

Contribution

The paper introduces PIP-Net, a novel recurrent and attention-based model for pedestrian intention prediction, along with the Urban-PIP dataset for real-world multi-camera scenarios.

Findings

Outperforms state-of-the-art models in pedestrian intention prediction

Predicts crossing intentions up to 4 seconds in advance

Enhances scene understanding with multi-camera and depth features

Abstract

Accurate pedestrian intention prediction (PIP) by Autonomous Vehicles (AVs) is one of the current research challenges in this field. In this article, we introduce PIP-Net, a novel framework designed to predict pedestrian crossing intentions by AVs in real-world urban scenarios. We offer two variants of PIP-Net designed for different camera mounts and setups. Leveraging both kinematic data and spatial features from the driving scene, the proposed model employs a recurrent and temporal attention-based solution, outperforming state-of-the-art performance. To enhance the visual representation of road users and their proximity to the ego vehicle, we introduce a categorical depth feature map, combined with a local motion flow feature, providing rich insights into the scene dynamics. Additionally, we explore the impact of expanding the camera's field of view, from one to three cameras surrounding the ego vehicle, leading to an enhancement in the model's contextual perception. Depending on the traffic scenario and road environment, the model excels in predicting pedestrian crossing intentions up to 4 seconds in advance, which is a breakthrough in current research studies in pedestrian intention prediction. Finally, for the first time, we present the Urban-PIP dataset, a customised pedestrian intention prediction dataset, with multi-camera annotations in real-world automated driving scenarios.