SRA-LSTM: Social Relationship Attention LSTM for Human Trajectory Prediction

TL;DR

This paper introduces SRA-LSTM, a novel model that incorporates social relationship attention to improve pedestrian trajectory prediction in surveillance videos, outperforming existing methods.

Contribution

The paper proposes a social relationship encoder and attention mechanism within LSTM to better model social influences on pedestrian movement.

Findings

SRA-LSTM achieves superior accuracy on ETH and UCY datasets.

Social relationship attention improves trajectory prediction performance.

Contrast experiments validate the effectiveness of the social relationship attention mechanism.

Abstract

Pedestrian trajectory prediction for surveillance video is one of the important research topics in the field of computer vision and a key technology of intelligent surveillance systems. Social relationship among pedestrians is a key factor influencing pedestrian walking patterns but was mostly ignored in the literature. Pedestrians with different social relationships play different roles in the motion decision of target pedestrian. Motivated by this idea, we propose a Social Relationship Attention LSTM (SRA-LSTM) model to predict future trajectories. We design a social relationship encoder to obtain the representation of their social relationship through the relative position between each pair of pedestrians. Afterwards, the social relationship feature and latent movements are adopted to acquire the social relationship attention of this pair of pedestrians. Social interaction modeling is achieved by utilizing social relationship attention to aggregate movement information from neighbor pedestrians. Experimental results on two public walking pedestrian video datasets (ETH and UCY), our model achieves superior performance compared with state-of-the-art methods. Contrast experiments with other attention methods also demonstrate the effectiveness of social relationship attention.