"If you could see me through my eyes": Predicting Pedestrian Perception

TL;DR

This paper introduces a machine learning approach using neural networks to predict pedestrians' future visual perceptions, aiding autonomous vehicles in understanding and anticipating pedestrian behavior for enhanced safety.

Contribution

It presents a novel neural network-based framework trained on synthetic data to simulate pedestrian perception and behavior prediction from a vehicle's perspective.

Findings

Accurately predicts pedestrian perceptions within relevant time horizons.

Networks can be used to simulate pedestrian behavior from the autonomous vehicle's viewpoint.

Potential for future training with real-world video data.

Abstract

Pedestrians are particularly vulnerable road users in urban traffic. With the arrival of autonomous driving, novel technologies can be developed specifically to protect pedestrians. We propose a machine learning toolchain to train artificial neural networks as models of pedestrian behavior. In a preliminary study, we use synthetic data from simulations of a specific pedestrian crossing scenario to train a variational autoencoder and a long short-term memory network to predict a pedestrian's future visual perception. We can accurately predict a pedestrian's future perceptions within relevant time horizons. By iteratively feeding these predicted frames into these networks, they can be used as simulations of pedestrians as indicated by our results. Such trained networks can later be used to predict pedestrian behaviors even from the perspective of the autonomous car. Another future extension will be to re-train these networks with real-world video data.