MoReFun: Past-Movement Guided Motion Representation Learning for Future Motion Prediction and Understanding

TL;DR

MoReFun introduces a two-stage self-supervised learning framework for 3D human motion prediction that improves accuracy and motion understanding by decoupling representation learning from prediction and focusing on dynamic joints.

Contribution

The paper proposes a novel two-stage self-supervised framework with velocity-based masking to enhance motion representation learning and prediction accuracy.

Findings

Reduces average prediction errors by 8.8% over state-of-the-art methods.

Effectively captures complex motion dynamics and dependencies.

Achieves competitive motion understanding performance.

Abstract

3D human motion prediction aims to generate coherent future motions from observed sequences, yet existing end-to-end regression frameworks often fail to capture complex dynamics and tend to produce temporally inconsistent or static predictions-a limitation rooted in representation shortcutting, where models rely on superficial cues rather than learning meaningful motion structure. We propose a two-stage self-supervised framework that decouples representation learning from prediction. In the pretraining stage, the model performs unified past-future self-reconstruction, reconstructing the past sequence while recovering masked joints in the future sequence under full historical guidance. A velocity-based masking strategy selects highly dynamic joints, forcing the model to focus on informative motion components and internalize the statistical dependencies between past and future states without regression interference. In the fine-tuning stage, the pretrained model predicts the entire future sequence, now treated as fully masked, and is further equipped with a lightweight future-text prediction head for joint optimization of low-level motion prediction and high-level motion understanding. Experiments on Human3.6M, 3DPW, and AMASS show that our method reduces average prediction errors by 8.8% over state-of-the-art methods while achieving competitive future-motion understanding performance compared to LLM-based models. Code is available at: https://github.com/JunyuShi02/MoReFun