Geometry-Contrastive Transformer for Generalized 3D Pose Transfer

TL;DR

This paper introduces a geometry-contrastive Transformer for 3D pose transfer that effectively detects and utilizes geometric inconsistencies across meshes, achieving state-of-the-art results and strong generalization to unseen datasets.

Contribution

It proposes a novel geometry-contrastive Transformer with a geodesic contrastive loss and regularization, advancing 3D pose transfer accuracy and robustness across diverse datasets.

Findings

State-of-the-art performance on SMPL-NPT, FAUST, and SMG-3D datasets.

Robust generalization to meshes from unknown spaces.

Effective regional geometric-inconsistency learning.

Abstract

We present a customized 3D mesh Transformer model for the pose transfer task. As the 3D pose transfer essentially is a deformation procedure dependent on the given meshes, the intuition of this work is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. Specifically, we propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies across the given meshes. Moreover, locally, a simple yet efficient central geodesic contrastive loss is further proposed to improve the regional geometric-inconsistency learning. At last, we present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task towards unknown spaces. The massive experimental results prove the efficacy of our approach by showing state-of-the-art quantitative performances on SMPL-NPT, FAUST and our new proposed dataset SMG-3D datasets, as well as promising qualitative results on MG-cloth and SMAL datasets. It's demonstrated that our method can achieve robust 3D pose transfer and be generalized to challenging meshes from unknown spaces on cross-dataset tasks. The code and dataset are made available. Code is available: https://github.com/mikecheninoulu/CGT.