History Repeats Itself: Human Motion Prediction via Motion Attention

TL;DR

This paper introduces a novel attention-based feed-forward network for human motion prediction that explicitly models the repeating nature of human motions, achieving state-of-the-art results on multiple datasets.

Contribution

The paper proposes a motion attention mechanism and a graph convolutional network to better exploit long-term motion patterns for improved prediction accuracy.

Findings

Achieves state-of-the-art results on Human3.6M, AMASS, and 3DPW datasets.

Effectively models both periodic and non-periodic human actions.

Outperforms existing methods in human motion prediction tasks.

Abstract

Human motion prediction aims to forecast future human poses given a past motion. Whether based on recurrent or feed-forward neural networks, existing methods fail to model the observation that human motion tends to repeat itself, even for complex sports actions and cooking activities. Here, we introduce an attention-based feed-forward network that explicitly leverages this observation. In particular, instead of modeling frame-wise attention via pose similarity, we propose to extract motion attention to capture the similarity between the current motion context and the historical motion sub-sequences. Aggregating the relevant past motions and processing the result with a graph convolutional network allows us to effectively exploit motion patterns from the long-term history to predict the future poses. Our experiments on Human3.6M, AMASS and 3DPW evidence the benefits of our approach for both periodical and non-periodical actions. Thanks to our attention model, it yields state-of-the-art results on all three datasets. Our code is available at https://github.com/wei-mao-2019/HisRepItself.