Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion

TL;DR

This paper introduces a new task and dataset for predicting pedestrian stop and go behaviors in urban traffic, proposing a hybrid model that fuses multiple features to improve robustness in dynamic scenarios.

Contribution

The paper presents the first benchmark dataset for pedestrian stop and go forecasting and a novel hybrid model that combines pedestrian-specific and scene features from multiple modalities.

Findings

The hybrid model outperforms baselines on the TRANS dataset.

Explicit modeling of stop and go behaviors improves prediction robustness.

The dataset enables focused research on non-linear pedestrian motion transitions.

Abstract

Forecasting pedestrians' future motions is essential for autonomous driving systems to safely navigate in urban areas. However, existing prediction algorithms often overly rely on past observed trajectories and tend to fail around abrupt dynamic changes, such as when pedestrians suddenly start or stop walking. We suggest that predicting these highly non-linear transitions should form a core component to improve the robustness of motion prediction algorithms. In this paper, we introduce the new task of pedestrian stop and go forecasting. Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic. We build it from several existing datasets annotated with pedestrians' walking motions, in order to have various scenarios and behaviors. We also propose a novel hybrid model that leverages pedestrian-specific and scene features from several modalities, both video sequences and high-level attributes, and gradually fuses them to integrate multiple levels of context. We evaluate our model and several baselines on TRANS, and set a new benchmark for the community to work on pedestrian stop and go forecasting.