Action-conditioned On-demand Motion Generation

TL;DR

This paper introduces ODMO, a novel framework for generating realistic, diverse, and customizable 3D human motions conditioned on action types, with innovative hierarchical encoding and decoding strategies.

Contribution

The paper presents a new motion generation framework with on-demand customization capabilities, including mode discovery, interpolation, and trajectory control, enabled by hierarchical encoder-decoder architecture.

Findings

Outperforms state-of-the-art on multiple datasets.

Demonstrates first-known customization features.

Provides qualitative and quantitative validation.

Abstract

We propose a novel framework, On-Demand MOtion Generation (ODMO), for generating realistic and diverse long-term 3D human motion sequences conditioned only on action types with an additional capability of customization. ODMO shows improvements over SOTA approaches on all traditional motion evaluation metrics when evaluated on three public datasets (HumanAct12, UESTC, and MoCap). Furthermore, we provide both qualitative evaluations and quantitative metrics demonstrating several first-known customization capabilities afforded by our framework, including mode discovery, interpolation, and trajectory customization. These capabilities significantly widen the spectrum of potential applications of such motion generation models. The novel on-demand generative capabilities are enabled by innovations in both the encoder and decoder architectures: (i) Encoder: Utilizing contrastive learning in low-dimensional latent space to create a hierarchical embedding of motion sequences, where not only the codes of different action types form different groups, but within an action type, codes of similar inherent patterns (motion styles) cluster together, making them readily discoverable; (ii) Decoder: Using a hierarchical decoding strategy where the motion trajectory is reconstructed first and then used to reconstruct the whole motion sequence. Such an architecture enables effective trajectory control. Our code is released on the Github page: https://github.com/roychowdhuryresearch/ODMO