MAPConNet: Self-supervised 3D Pose Transfer with Mesh and Point Contrastive Learning

TL;DR

This paper introduces MAPConNet, a self-supervised framework for 3D pose transfer that leverages contrastive learning in latent space, enabling effective transfer without requiring correspondence labels and achieving state-of-the-art results.

Contribution

The paper proposes a novel self-supervised approach using contrastive learning for 3D pose transfer, eliminating the need for correspondence labels and enabling training in various supervision settings.

Findings

Achieves state-of-the-art results in supervised 3D pose transfer.

Performs comparably in unsupervised and semi-supervised settings.

Generalizes well to unseen human and animal data with complex topologies.

Abstract

3D pose transfer is a challenging generation task that aims to transfer the pose of a source geometry onto a target geometry with the target identity preserved. Many prior methods require keypoint annotations to find correspondence between the source and target. Current pose transfer methods allow end-to-end correspondence learning but require the desired final output as ground truth for supervision. Unsupervised methods have been proposed for graph convolutional models but they require ground truth correspondence between the source and target inputs. We present a novel self-supervised framework for 3D pose transfer which can be trained in unsupervised, semi-supervised, or fully supervised settings without any correspondence labels. We introduce two contrastive learning constraints in the latent space: a mesh-level loss for disentangling global patterns including pose and identity, and a point-level loss for discriminating local semantics. We demonstrate quantitatively and qualitatively that our method achieves state-of-the-art results in supervised 3D pose transfer, with comparable results in unsupervised and semi-supervised settings. Our method is also generalisable to unseen human and animal data with complex topologies.