Flexible Motion In-betweening with Diffusion Models

TL;DR

This paper introduces CondMDI, a diffusion model-based approach for flexible, diverse, and high-quality motion in-betweening guided by keyframes and text, improving upon prior methods in animation.

Contribution

It presents a unified diffusion model capable of handling arbitrary keyframe constraints and text guidance for motion in-betweening, a novel approach in character animation.

Findings

CondMDI produces diverse, coherent motions aligned with keyframes.

The model outperforms existing methods on the HumanML3D dataset.

Guidance and imputation techniques enhance inference flexibility.

Abstract

Motion in-betweening, a fundamental task in character animation, consists of generating motion sequences that plausibly interpolate user-provided keyframe constraints. It has long been recognized as a labor-intensive and challenging process. We investigate the potential of diffusion models in generating diverse human motions guided by keyframes. Unlike previous inbetweening methods, we propose a simple unified model capable of generating precise and diverse motions that conform to a flexible range of user-specified spatial constraints, as well as text conditioning. To this end, we propose Conditional Motion Diffusion In-betweening (CondMDI) which allows for arbitrary dense-or-sparse keyframe placement and partial keyframe constraints while generating high-quality motions that are diverse and coherent with the given keyframes. We evaluate the performance of CondMDI on the text-conditioned HumanML3D dataset and demonstrate the versatility and efficacy of diffusion models for keyframe in-betweening. We further explore the use of guidance and imputation-based approaches for inference-time keyframing and compare CondMDI against these methods.