Learning Semantic Latent Directions for Accurate and Controllable Human Motion Prediction

TL;DR

This paper introduces Semantic Latent Directions (SLD), a novel approach that constrains the latent space in human motion prediction models to improve accuracy, realism, and controllability of predicted motions, while also enhancing diversity.

Contribution

The paper proposes SLD, a new method that learns meaningful motion semantics through orthogonal latent directions, enabling more accurate and controllable human motion predictions.

Findings

SLD improves motion prediction accuracy on benchmark datasets.

The method offers controllable and diverse motion predictions.

Experiments demonstrate superior performance over existing models.

Abstract

In the realm of stochastic human motion prediction (SHMP), researchers have often turned to generative models like GANS, VAEs and diffusion models. However, most previous approaches have struggled to accurately predict motions that are both realistic and coherent with past motion due to a lack of guidance on the latent distribution. In this paper, we introduce Semantic Latent Directions (SLD) as a solution to this challenge, aiming to constrain the latent space to learn meaningful motion semantics and enhance the accuracy of SHMP. SLD defines a series of orthogonal latent directions and represents the hypothesis of future motion as a linear combination of these directions. By creating such an information bottleneck, SLD excels in capturing meaningful motion semantics, thereby improving the precision of motion predictions. Moreover, SLD offers controllable prediction capabilities by adjusting the coefficients of the latent directions during the inference phase. Expanding on SLD, we introduce a set of motion queries to enhance the diversity of predictions. By aligning these motion queries with the SLD space, SLD is further promoted to more accurate and coherent motion predictions. Through extensive experiments conducted on widely used benchmarks, we showcase the superiority of our method in accurately predicting motions while maintaining a balance of realism and diversity. Our code and pretrained models are available at https://github.com/GuoweiXu368/SLD-HMP.