A Plug-and-Play Multi-Criteria Guidance for Diverse In-Betweening Human Motion Generation

TL;DR

This paper introduces MCG-IMM, a plug-and-play multi-criteria guidance method that enhances diversity and smoothness in in-betweening human motion generation using pretrained models without extra parameters.

Contribution

It reformulates the sampling of pretrained generative models as a multi-criteria optimization problem, improving diversity and smoothness in generated motions.

Findings

Achieves state-of-the-art results on four human motion datasets.

Compatible with various generative model families.

Effectively enhances diversity without additional parameters.

Abstract

In-betweening human motion generation aims to synthesize intermediate motions that transition between user-specified keyframes. In addition to maintaining smooth transitions, a crucial requirement of this task is to generate diverse motion sequences. It is still challenging to maintain diversity, particularly when it is necessary for the motions within a generated batch sampling to differ meaningfully from one another due to complex motion dynamics. In this paper, we propose a novel method, termed the Multi-Criteria Guidance with In-Betweening Motion Model (MCG-IMM), for in-betweening human motion generation. A key strength of MCG-IMM lies in its plug-and-play nature: it enhances the diversity of motions generated by pretrained models without introducing additional parameters This is achieved by providing a sampling process of pretrained generative models with multi-criteria guidance. Specifically, MCG-IMM reformulates the sampling process of pretrained generative model as a multi-criteria optimization problem, and introduces an optimization process to explore motion sequences that satisfy multiple criteria, e.g., diversity and smoothness. Moreover, our proposed plug-and-play multi-criteria guidance is compatible with different families of generative models, including denoised diffusion probabilistic models, variational autoencoders, and generative adversarial networks. Experiments on four popular human motion datasets demonstrate that MCG-IMM consistently state-of-the-art methods in in-betweening motion generation task.