UnityGraph: Unified Learning of Spatio-temporal features for Multi-person Motion Prediction

TL;DR

UnityGraph introduces a hypergraph-based approach to unify spatial and temporal features in multi-person motion prediction, improving coherence and coupling for more accurate predictions.

Contribution

The paper proposes UnityGraph, a hypergraph structure that models spatio-temporal features as a whole, enabling better fusion and coherence in multi-person motion prediction.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively models spatio-temporal dynamics as a unified graph.

Demonstrates improved coherence and coupling in motion prediction.

Abstract

Multi-person motion prediction is a complex and emerging field with significant real-world applications. Current state-of-the-art methods typically adopt dual-path networks to separately modeling spatial features and temporal features. However, the uncertain compatibility of the two networks brings a challenge for spatio-temporal features fusion and violate the spatio-temporal coherence and coupling of human motions by nature. To address this issue, we propose a novel graph structure, UnityGraph, which treats spatio-temporal features as a whole, enhancing model coherence and coupling.spatio-temporal features as a whole, enhancing model coherence and coupling. Specifically, UnityGraph is a hypervariate graph based network. The flexibility of the hypergraph allows us to consider the observed motions as graph nodes. We then leverage hyperedges to bridge these nodes for exploring spatio-temporal features. This perspective considers spatio-temporal dynamics unitedly and reformulates multi-person motion prediction into a problem on a single graph. Leveraging the dynamic message passing based on this hypergraph, our model dynamically learns from both types of relations to generate targeted messages that reflect the relevance among nodes. Extensive experiments on several datasets demonstrates that our method achieves state-of-the-art performance, confirming its effectiveness and innovative design.