Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic

TL;DR

This paper introduces a novel approach for stochastic human motion prediction that utilizes memory banks for action transition and characteristics, improving the realism and consistency of predicted motions.

Contribution

The paper proposes two memory banks, STAB and ACB, and an adaptive attention strategy to enhance motion prediction accuracy and handle transition and characteristic learning challenges.

Findings

Outperforms previous state-of-the-art on four datasets

Effectively models action transitions with soft searching

Accurately captures action characteristics for better predictions

Abstract

Action-driven stochastic human motion prediction aims to generate future motion sequences of a pre-defined target action based on given past observed sequences performing non-target actions. This task primarily presents two challenges. Firstly, generating smooth transition motions is hard due to the varying transition speeds of different actions. Secondly, the action characteristic is difficult to be learned because of the similarity of some actions. These issues cause the predicted results to be unreasonable and inconsistent. As a result, we propose two memory banks, the Soft-transition Action Bank (STAB) and Action Characteristic Bank (ACB), to tackle the problems above. The STAB stores the action transition information. It is equipped with the novel soft searching approach, which encourages the model to focus on multiple possible action categories of observed motions. The ACB records action characteristic, which produces more prior information for predicting certain actions. To fuse the features retrieved from the two banks better, we further propose the Adaptive Attention Adjustment (AAA) strategy. Extensive experiments on four motion prediction datasets demonstrate that our approach consistently outperforms the previous state-of-the-art. The demo and code are available at https://hyqlat.github.io/STABACB.github.io/.