Soft Attention: Does it Actually Help to Learn Social Interactions in Pedestrian Trajectory Prediction?

TL;DR

This paper critically examines whether soft-attention mechanisms in pedestrian trajectory prediction models genuinely utilize social information, finding that these models tend to ignore social cues despite their design to incorporate them.

Contribution

The study reveals that state-of-the-art models with soft-attention mechanisms do not actually leverage social information during prediction, challenging assumptions about their effectiveness.

Findings

Models trained with social information perform similarly to those trained with noise.

Soft-attention mechanisms are often ignored by the models.

Models can shut down social modules without affecting performance.

Abstract

We consider the problem of predicting the future path of a pedestrian using its motion history and the motion history of the surrounding pedestrians, called social information. Since the seminal paper on Social-LSTM, deep-learning has become the main tool used to model the impact of social interactions on a pedestrian's motion. The demonstration that these models can learn social interactions relies on an ablative study of these models. The models are compared with and without their social interactions module on two standard metrics, the Average Displacement Error and Final Displacement Error. Yet, these complex models were recently outperformed by a simple constant-velocity approach. This questions if they actually allow to model social interactions as well as the validity of the proof. In this paper, we focus on the deep-learning models with a soft-attention mechanism for social interaction modeling and study whether they use social information at prediction time. We conduct two experiments across four state-of-the-art approaches on the ETH and UCY datasets, which were also used in previous work. First, the models are trained by replacing the social information with random noise and compared to model trained with actual social information. Second, we use a gating mechanism along with a $L_0$ penalty, allowing models to shut down their inner components. The models consistently learn to prune their soft-attention mechanism. For both experiments, neither the course of the convergence nor the prediction performance were altered. This demonstrates that the soft-attention mechanism and therefore the social information are ignored by the models.