SOFTooth: Semantics-Enhanced Order-Aware Fusion for Tooth Instance Segmentation

TL;DR

SOFTooth introduces a novel 2D-3D fusion framework that leverages semantics from 2D models to improve 3D tooth segmentation accuracy, especially in complex cases with missing or crowded teeth.

Contribution

It proposes a semantics-enhanced, order-aware fusion method that effectively transfers 2D semantics to 3D segmentation without requiring 2D mask supervision.

Findings

Achieves state-of-the-art accuracy on 3DTeethSeg'22

Improves segmentation of third molars and crowded dentitions

Reduces boundary leakage and center drift in 3D segmentation

Abstract

Three-dimensional (3D) tooth instance segmentation remains challenging due to crowded arches, ambiguous tooth-gingiva boundaries, missing teeth, and rare yet clinically important third molars. Native 3D methods relying on geometric cues often suffer from boundary leakage, center drift, and inconsistent tooth identities, especially for minority classes and complex anatomies. Meanwhile, 2D foundation models such as the Segment Anything Model (SAM) provide strong boundary-aware semantics, but directly applying them in 3D is impractical in clinical workflows. To address these issues, we propose SOFTooth, a semantics-enhanced, order-aware 2D-3D fusion framework that leverages frozen 2D semantics without explicit 2D mask supervision. First, a point-wise residual gating module injects occlusal-view SAM embeddings into 3D point features to refine tooth-gingiva and inter-tooth boundaries. Second, a center-guided mask refinement regularizes consistency between instance masks and geometric centroids, reducing center drift. Furthermore, an order-aware Hungarian matching strategy integrates anatomical tooth order and center distance into similarity-based assignment, ensuring coherent labeling even under missing or crowded dentitions. On 3DTeethSeg'22, SOFTooth achieves state-of-the-art overall accuracy and mean IoU, with clear gains on cases involving third molars, demonstrating that rich 2D semantics can be effectively transferred to 3D tooth instance segmentation without 2D fine-tuning.