Controlling Character Motions without Observable Driving Source

TL;DR

This paper introduces a novel framework for generating diverse, long, and realistic character motion sequences without relying on external driving sources, overcoming key challenges like error accumulation and lack of diversity.

Contribution

It proposes a systematic approach combining VQ-VAE and reinforcement learning to control motion generation without observable driving sources, with potential for generalization.

Findings

Outperforms existing baselines significantly

Effectively mitigates out-of-distribution issues

Produces diverse and natural motion sequences

Abstract

How to generate diverse, life-like, and unlimited long head/body sequences without any driving source? We argue that this under-investigated research problem is non-trivial at all, and has unique technical challenges behind it. Without semantic constraints from the driving sources, using the standard autoregressive model to generate infinitely long sequences would easily result in 1) out-of-distribution (OOD) issue due to the accumulated error, 2) insufficient diversity to produce natural and life-like motion sequences and 3) undesired periodic patterns along the time. To tackle the above challenges, we propose a systematic framework that marries the benefits of VQ-VAE and a novel token-level control policy trained with reinforcement learning using carefully designed reward functions. A high-level prior model can be easily injected on top to generate unlimited long and diverse sequences. Although we focus on no driving sources now, our framework can be generalized for controlled synthesis with explicit driving sources. Through comprehensive evaluations, we conclude that our proposed framework can address all the above-mentioned challenges and outperform other strong baselines very significantly.