Best Practices for 2-Body Pose Forecasting

TL;DR

This paper reviews and identifies best practices for 2-body human pose forecasting, introducing a novel initialization method that improves performance and stability, achieving a 21.9% improvement over the state-of-the-art.

Contribution

It provides an in-depth assessment of single-person forecasting practices for 2-body interactions and proposes a new initialization procedure for interaction parameters.

Findings

Frequency input representations improve forecasting accuracy.

Space-time separable and learnable interaction adjacencies are effective.

The proposed methods improve performance by 21.9% over previous state-of-the-art.

Abstract

The task of collaborative human pose forecasting stands for predicting the future poses of multiple interacting people, given those in previous frames. Predicting two people in interaction, instead of each separately, promises better performance, due to their body-body motion correlations. But the task has remained so far primarily unexplored. In this paper, we review the progress in human pose forecasting and provide an in-depth assessment of the single-person practices that perform best for 2-body collaborative motion forecasting. Our study confirms the positive impact of frequency input representations, space-time separable and fully-learnable interaction adjacencies for the encoding GCN and FC decoding. Other single-person practices do not transfer to 2-body, so the proposed best ones do not include hierarchical body modeling or attention-based interaction encoding. We further contribute a novel initialization procedure for the 2-body spatial interaction parameters of the encoder, which benefits performance and stability. Altogether, our proposed 2-body pose forecasting best practices yield a performance improvement of 21.9% over the state-of-the-art on the most recent ExPI dataset, whereby the novel initialization accounts for 3.5%. See our project page at https://www.pinlab.org/bestpractices2body