Semantic-Aware Implicit Template Learning via Part Deformation Consistency

TL;DR

This paper introduces a semantic-aware implicit template learning framework that uses part deformation consistency and semantic priors to improve shape correspondence tasks involving high structural variability.

Contribution

It proposes a novel semantic-aware deformation code and regularizations to enhance implicit template learning for more plausible shape deformations.

Findings

Outperforms baselines in keypoint, part label, and texture transfer tasks.

Shows larger performance gains in challenging settings.

Qualitative analyses confirm the effectiveness of semantic-aware deformation.

Abstract

Learning implicit templates as neural fields has recently shown impressive performance in unsupervised shape correspondence. Despite the success, we observe current approaches, which solely rely on geometric information, often learn suboptimal deformation across generic object shapes, which have high structural variability. In this paper, we highlight the importance of part deformation consistency and propose a semantic-aware implicit template learning framework to enable semantically plausible deformation. By leveraging semantic prior from a self-supervised feature extractor, we suggest local conditioning with novel semantic-aware deformation code and deformation consistency regularizations regarding part deformation, global deformation, and global scaling. Our extensive experiments demonstrate the superiority of the proposed method over baselines in various tasks: keypoint transfer, part label transfer, and texture transfer. More interestingly, our framework shows a larger performance gain under more challenging settings. We also provide qualitative analyses to validate the effectiveness of semantic-aware deformation. The code is available at https://github.com/mlvlab/PDC.