TrajeVAE: Controllable Human Motion Generation from Trajectories

TL;DR

TrajeVAE introduces a transformer-based framework for controllable 3D human motion generation from multiple trajectories, enabling more flexible and accurate animations compared to existing methods.

Contribution

The paper presents TrajeVAE, a novel transformer-like architecture that reformulates human motion prediction as pose completion, allowing multi-trajectory control and improved accuracy.

Findings

Outperforms trajectory-based methods in accuracy

Can generate plausible motions from a single initial pose

Handles multiple trajectories for fine-grained control

Abstract

The creation of plausible and controllable 3D human motion animations is a long-standing problem that requires a manual intervention of skilled artists. Current machine learning approaches can semi-automate the process, however, they are limited in a significant way: they can handle only a single trajectory of the expected motion that precludes fine-grained control over the output. To mitigate that issue, we reformulate the problem of future pose prediction into pose completion in space and time where multiple trajectories are represented as poses with missing joints. We show that such a framework can generalize to other neural networks designed for future pose prediction. Once trained in this framework, a model is capable of predicting sequences from any number of trajectories. We propose a novel transformer-like architecture, TrajeVAE, that builds on this idea and provides a versatile framework for 3D human animation. We demonstrate that TrajeVAE offers better accuracy than the trajectory-based reference approaches and methods that base their predictions on past poses. We also show that it can predict reasonable future poses even if provided only with an initial pose.