Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic

TL;DR

This paper introduces a novel approach for long-term human motion prediction by modeling motion context and enhancing motion dynamics, leading to improved forecasting accuracy and enabling motion transfer based on activity labels.

Contribution

It proposes a motion context modeling method with a modified highway unit and a gram matrix loss for better long-term prediction and motion transfer.

Findings

Outperforms state-of-the-art methods in long-term motion prediction

Effectively models motion context for improved accuracy

Enables human motion transfer using activity labels

Abstract

Human motion prediction aims at generating future frames of human motion based on an observed sequence of skeletons. Recent methods employ the latest hidden states of a recurrent neural network (RNN) to encode the historical skeletons, which can only address short-term prediction. In this work, we propose a motion context modeling by summarizing the historical human motion with respect to the current prediction. A modified highway unit (MHU) is proposed for efficiently eliminating motionless joints and estimating next pose given the motion context. Furthermore, we enhance the motion dynamic by minimizing the gram matrix loss for long-term motion prediction. Experimental results show that the proposed model can promisingly forecast the human future movements, which yields superior performances over related state-of-the-art approaches. Moreover, specifying the motion context with the activity labels enables our model to perform human motion transfer.