Action2Motion: Conditioned Generation of 3D Human Motions

TL;DR

This paper introduces Action2Motion, a model that generates diverse and realistic 3D human motion sequences conditioned on specific actions, using Lie algebra representations and a temporal VAE, validated on multiple datasets.

Contribution

It presents a novel approach combining Lie algebra and a temporal VAE for diverse, action-conditioned 3D human motion generation, along with a new dataset HumanAct12.

Findings

Effective generation of diverse human motions

Successful conditioning on action types

Validated on multiple datasets

Abstract

Action recognition is a relatively established task, where givenan input sequence of human motion, the goal is to predict its ac-tion category. This paper, on the other hand, considers a relativelynew problem, which could be thought of as an inverse of actionrecognition: given a prescribed action type, we aim to generateplausible human motion sequences in 3D. Importantly, the set ofgenerated motions are expected to maintain itsdiversityto be ableto explore the entire action-conditioned motion space; meanwhile,each sampled sequence faithfully resembles anaturalhuman bodyarticulation dynamics. Motivated by these objectives, we followthe physics law of human kinematics by adopting the Lie Algebratheory to represent thenaturalhuman motions; we also propose atemporal Variational Auto-Encoder (VAE) that encourages adiversesampling of the motion space. A new 3D human motion dataset, HumanAct12, is also constructed. Empirical experiments overthree distinct human motion datasets (including ours) demonstratethe effectiveness of our approach.