Generative Motion In-betweening by Diffusion over Continuous Implicit Representations

TL;DR

This paper introduces a new method using latent diffusion models and implicit neural representations to generate smooth, realistic motion in-between frames from sparse keyframe data, improving quality and diversity.

Contribution

It presents a novel pipeline and sampling strategy that effectively reconstructs plausible motions from minimal keyframe information, addressing key limitations of prior methods.

Findings

Significantly improves motion generation quality with few keyframes

Ensures both keyframe accuracy and motion diversity

Demonstrates superior performance over existing approaches

Abstract

Recent advances in generative models have yielded impressive progress on motion in-betweening, allowing for more complex, varied, and realistic motion transitions. However, recent methods still exhibit noticeable limitations in preserving keyframe information and ensuring motion continuity. In this paper, we propose a novel pipeline and sampling optimization strategy for latent diffusion models (LDM) based on motion implicit neural representations (INR). By establishing a mapping between INR and sparse spatial or temporal information within latent diffusion, our model can sample the INR parameters from extremely sparse and ambiguous keyframe data and reconstruct plausible and smooth motions from the manifold. Our experiments demonstrate the superior performance of our model, which significantly improves motion generation quality in scenarios with few keyframes while ensuring both keyframe accuracy and diversity of in-between motions.