Social-Pose: Enhancing Trajectory Prediction with Human Body Pose

TL;DR

This paper introduces Social-pose, an attention-based pose encoder that improves human trajectory prediction by leveraging human body poses, demonstrating enhanced accuracy across multiple models and datasets for autonomous driving safety.

Contribution

The paper presents a novel pose-based trajectory prediction method that can be integrated into existing models, showing consistent improvements over state-of-the-art approaches.

Findings

Pose-based models outperform Cartesian location models in trajectory prediction.

Using 2D and 3D poses both improve prediction accuracy.

The approach is effective even with noisy pose data and in robot navigation scenarios.

Abstract

Accurate human trajectory prediction is one of the most crucial tasks for autonomous driving, ensuring its safety. Yet, existing models often fail to fully leverage the visual cues that humans subconsciously communicate when navigating the space. In this work, we study the benefits of predicting human trajectories using human body poses instead of solely their Cartesian space locations in time. We propose `Social-pose', an attention-based pose encoder that effectively captures the poses of all humans in a scene and their social relations. Our method can be integrated into various trajectory prediction architectures. We have conducted extensive experiments on state-of-the-art models (based on LSTM, GAN, MLP, and Transformer), and showed improvements over all of them on synthetic (Joint Track Auto) and real (Human3.6M, Pedestrians and Cyclists in Road Traffic, and JRDB) datasets. We also explored the advantages of using 2D versus 3D poses, as well as the effect of noisy poses and the application of our pose-based predictor in robot navigation scenarios.