Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation

TL;DR

This paper introduces a novel framework for 3D motion interpolation from sparse keyframes, leveraging latent motion manifolds to improve continuity and accuracy in generated motions.

Contribution

It proposes a two-stage framework that models latent motion manifolds with keyframe constraints, enhancing intermediate token generation for motion interpolation.

Findings

Superior interpolation accuracy on LaFAN1 and CMU Mocap datasets.

High visual similarity to ground truth motions.

Effective modeling of continuous motion representations.

Abstract

Deriving sophisticated 3D motions from sparse keyframes is a particularly challenging problem, due to continuity and exceptionally skeletal precision. The action features are often derivable accurately from the full series of keyframes, and thus, leveraging the global context with transformers has been a promising data-driven embedding approach. However, existing methods are often with inputs of interpolated intermediate frame for continuity using basic interpolation methods with keyframes, which result in a trivial local minimum during training. In this paper, we propose a novel framework to formulate latent motion manifolds with keyframe-based constraints, from which the continuous nature of intermediate token representations is considered. Particularly, our proposed framework consists of two stages for identifying a latent motion subspace, i.e., a keyframe encoding stage and an intermediate token generation stage, and a subsequent motion synthesis stage to extrapolate and compose motion data from manifolds. Through our extensive experiments conducted on both the LaFAN1 and CMU Mocap datasets, our proposed method demonstrates both superior interpolation accuracy and high visual similarity to ground truth motions.