WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds

TL;DR

This paper introduces a novel phase manifold approach for cross-morphology motion alignment that learns shared motion representations across different character types without supervision, enabling applications like motion transfer and stylization.

Contribution

It proposes a vector quantized autoencoder to learn a shared phase manifold for multiple characters, facilitating unsupervised semantic and timing alignment of motions across different morphologies.

Findings

Successfully aligns motions of humans and dogs without supervision.

Enables effective motion retrieval, transfer, and stylization.

Demonstrates robustness across various applications.

Abstract

We present a new approach for understanding the periodicity structure and semantics of motion datasets, independently of the morphology and skeletal structure of characters. Unlike existing methods using an overly sparse high-dimensional latent, we propose a phase manifold consisting of multiple closed curves, each corresponding to a latent amplitude. With our proposed vector quantized periodic autoencoder, we learn a shared phase manifold for multiple characters, such as a human and a dog, without any supervision. This is achieved by exploiting the discrete structure and a shallow network as bottlenecks, such that semantically similar motions are clustered into the same curve of the manifold, and the motions within the same component are aligned temporally by the phase variable. In combination with an improved motion matching framework, we demonstrate the manifold's capability of timing and semantics alignment in several applications, including motion retrieval, transfer and stylization. Code and pre-trained models for this paper are available at https://peizhuoli.github.io/walkthedog.