Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs

TL;DR

This paper introduces a novel stacked RNN architecture that fuses scene dynamics and visual features for improved pedestrian action anticipation in urban environments, enhancing autonomous vehicle safety.

Contribution

It presents a new multi-source feature fusion approach within stacked RNNs for predicting pedestrian crossing actions, outperforming existing recurrent models.

Findings

Higher prediction accuracy than alternative RNN architectures

Impact of observation length and feature types on performance

Data fusion strategies significantly affect prediction results

Abstract

One of the major challenges for autonomous vehicles in urban environments is to understand and predict other road users' actions, in particular, pedestrians at the point of crossing. The common approach to solving this problem is to use the motion history of the agents to predict their future trajectories. However, pedestrians exhibit highly variable actions most of which cannot be understood without visual observation of the pedestrians themselves and their surroundings. To this end, we propose a solution for the problem of pedestrian action anticipation at the point of crossing. Our approach uses a novel stacked RNN architecture in which information collected from various sources, both scene dynamics and visual features, is gradually fused into the network at different levels of processing. We show, via extensive empirical evaluations, that the proposed algorithm achieves a higher prediction accuracy compared to alternative recurrent network architectures. We conduct experiments to investigate the impact of the length of observation, time to event and types of features on the performance of the proposed method. Finally, we demonstrate how different data fusion strategies impact prediction accuracy.