Understanding Pedestrian-Vehicle Interactions with Vehicle Mounted Vision: An LSTM Model and Empirical Analysis

TL;DR

This paper introduces an LSTM-based model that predicts pedestrian movements by analyzing vehicle-mounted camera data, enhancing understanding of pedestrian-vehicle interactions in complex urban environments.

Contribution

It presents a novel data-driven LSTM model incorporating pedestrian head orientation and vehicle trajectories to improve pedestrian behavior prediction.

Findings

The LSTM model outperforms baseline trajectory-only methods.

Including head orientation improves prediction accuracy.

Empirical results are validated on real-world urban datasets.

Abstract

Pedestrians and vehicles often share the road in complex inner city traffic. This leads to interactions between the vehicle and pedestrians, with each affecting the other's motion. In order to create robust methods to reason about pedestrian behavior and to design interfaces of communication between self-driving cars and pedestrians we need to better understand such interactions. In this paper, we present a data-driven approach to implicitly model pedestrians' interactions with vehicles, to better predict pedestrian behavior. We propose a LSTM model that takes as input the past trajectories of the pedestrian and ego-vehicle, and pedestrian head orientation, and predicts the future positions of the pedestrian. Our experiments based on a real-world, inner city dataset captured with vehicle mounted cameras, show that the usage of such cues improve pedestrian prediction when compared to a baseline that purely uses the past trajectory of the pedestrian.

Figures