Joint-Relation Transformer for Multi-Person Motion Prediction

TL;DR

This paper introduces the Joint-Relation Transformer, which incorporates explicit relation information like distances and constraints to enhance multi-person motion prediction accuracy, outperforming existing transformer-based methods.

Contribution

The paper proposes a novel joint-relation fusion layer with relation-aware attention and supervision via future distance forecasting, improving interaction modeling in motion prediction.

Findings

Achieves 13.4% improvement on 900ms VIM metric.

Improves 3s MPJPE by 17.8% on CMU-Mpcap dataset.

Enhances interaction modeling with relation-aware attention.

Abstract

Multi-person motion prediction is a challenging problem due to the dependency of motion on both individual past movements and interactions with other people. Transformer-based methods have shown promising results on this task, but they miss the explicit relation representation between joints, such as skeleton structure and pairwise distance, which is crucial for accurate interaction modeling. In this paper, we propose the Joint-Relation Transformer, which utilizes relation information to enhance interaction modeling and improve future motion prediction. Our relation information contains the relative distance and the intra-/inter-person physical constraints. To fuse relation and joint information, we design a novel joint-relation fusion layer with relation-aware attention to update both features. Additionally, we supervise the relation information by forecasting future distance. Experiments show that our method achieves a 13.4% improvement of 900ms VIM on 3DPW-SoMoF/RC and 17.8%/12.0% improvement of 3s MPJPE on CMU-Mpcap/MuPoTS-3D dataset.