SGSoft: Learning Fused Semantic-Geometric Features for 3D Shape Correspondence via Template-Guided Soft Signals

TL;DR

SGSoft introduces a unified pipeline for learning dense 3D shape correspondences that is robust to structural variability and offers real-time inference, surpassing prior methods in accuracy and efficiency.

Contribution

The paper presents SGSoft, a novel intrinsic method that constructs geodesic correspondence fields and learns multimodal descriptors guided by semantic priors for stable, topology-invariant 3D shape correspondence.

Findings

Achieves state-of-the-art inter-category generalization.

Offers the best accuracy-efficiency trade-off among prior methods.

Enables near real-time inference without pre-alignment or post-refinement.

Abstract

Learning dense correspondences across deformable 3D shapes remains a long-standing challenge due to structural variability, non-isometric deformation, and inconsistent topology. Existing methods typically trade off generalization, geometric fidelity, and efficiency. We address this by proposing SGSoft, a unified intrinsic pipeline that (i) constructs a geodesic correspondence field on a canonical template, (ii) learns multimodal dense descriptors guided by pretrained semantic priors with this geodesic correspondence field supervision, (iii) retrieves dense correspondences in a single feed-forward pass via nearest-neighbor search in descriptor space. This formulation enables stable and topology-invariant supervision under large pose variation, structural differences, and remeshing. SGSoft achieves state-of-the-art inter-category generalization while offering the best accuracy-efficiency trade-off among prior methods. It also achieves near real-time inference without pre-alignment, pairwise optimization, or post-refinement. Learned descriptors can be transferred effectively to downstream tasks such as semantic segmentation and deformation transfer, establishing a scalable and deployment-ready paradigm for dense 3D correspondence.