Structure-Aware Motion Transfer with Deformable Anchor Model

TL;DR

This paper introduces the deformable anchor model (DAM), a structure-aware, unsupervised approach for motion transfer that automatically learns object structure without prior information, outperforming existing methods.

Contribution

The deformable anchor model (DAM) is a novel, unsupervised method that captures object structure for motion transfer without relying on prior structural information.

Findings

DAM achieves superior performance on benchmark datasets.

DAM effectively models complex object structures hierarchically.

Unsupervised learning of motion transfer with DAM outperforms state-of-the-art methods.

Abstract

Given a source image and a driving video depicting the same object type, the motion transfer task aims to generate a video by learning the motion from the driving video while preserving the appearance from the source image. In this paper, we propose a novel structure-aware motion modeling approach, the deformable anchor model (DAM), which can automatically discover the motion structure of arbitrary objects without leveraging their prior structure information. Specifically, inspired by the known deformable part model (DPM), our DAM introduces two types of anchors or keypoints: i) a number of motion anchors that capture both appearance and motion information from the source image and driving video; ii) a latent root anchor, which is linked to the motion anchors to facilitate better learning of the representations of the object structure information. Moreover, DAM can be further extended to a hierarchical version through the introduction of additional latent anchors to model more complicated structures. By regularizing motion anchors with latent anchor(s), DAM enforces the correspondences between them to ensure the structural information is well captured and preserved. Moreover, DAM can be learned effectively in an unsupervised manner. We validate our proposed DAM for motion transfer on different benchmark datasets. Extensive experiments clearly demonstrate that DAM achieves superior performance relative to existing state-of-the-art methods.