CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion

TL;DR

CoMusion introduces a diffusion-based, end-to-end framework for stochastic human motion prediction that ensures realistic, diverse, and consistent future pose sequences by combining Transformer and GCN modules.

Contribution

The paper presents a novel single-stage diffusion model with Transformer-GCN architecture for improved stochastic human motion prediction, addressing prior methods' inconsistency and complexity.

Findings

Outperforms prior methods on benchmark datasets

Produces more realistic and diverse motion predictions

Maintains consistency with observed human motion

Abstract

Stochastic Human Motion Prediction (HMP) aims to predict multiple possible future human pose sequences from observed ones. Most prior works learn motion distributions through encoding-decoding in the latent space, which does not preserve motion's spatial-temporal structure. While effective, these methods often require complex, multi-stage training and yield predictions that are inconsistent with the provided history and can be physically unrealistic. To address these issues, we propose CoMusion, a single-stage, end-to-end diffusion-based stochastic HMP framework. CoMusion is inspired from the insight that a smooth future pose initialization improves prediction performance, a strategy not previously utilized in stochastic models but evidenced in deterministic works. To generate such initialization, CoMusion's motion predictor starts with a Transformer-based network for initial reconstruction of corrupted motion. Then, a graph convolutional network (GCN) is employed to refine the prediction considering past observations in the discrete cosine transformation (DCT) space. Our method, facilitated by the Transformer-GCN module design and a proposed variance scheduler, excels in predicting accurate, realistic, and consistent motions, while maintaining appropriate diversity. Experimental results on benchmark datasets demonstrate that CoMusion surpasses prior methods across metrics, while demonstrating superior generation quality. Our Code is released at https://github.com/jsun57/CoMusion/ .