Learning to In-paint: Domain Adaptive Shape Completion for 3D Organ Segmentation

TL;DR

This paper introduces a novel shape learning method for 3D organ segmentation using in-painting with transformers, improving domain adaptation and segmentation accuracy across multiple datasets.

Contribution

It formulates shape learning as an in-painting task with transformer-based label mask completion and proposes a shape-aware self-distillation method for domain transfer.

Findings

Achieved at least 1.2 points improvement in Dice score

Effective in unsupervised domain adaptation scenarios

Improved segmentation of unseen organs and domains

Abstract

We aim at incorporating explicit shape information into current 3D organ segmentation models. Different from previous works, we formulate shape learning as an in-painting task, which is named Masked Label Mask Modeling (MLM). Through MLM, learnable mask tokens are fed into transformer blocks to complete the label mask of organ. To transfer MLM shape knowledge to target, we further propose a novel shape-aware self-distillation with both in-painting reconstruction loss and pseudo loss. Extensive experiments on five public organ segmentation datasets show consistent improvements over prior arts with at least 1.2 points gain in the Dice score, demonstrating the effectiveness of our method in challenging unsupervised domain adaptation scenarios including: (1) In-domain organ segmentation; (2) Unseen domain segmentation and (3) Unseen organ segmentation. We hope this work will advance shape analysis and geometric learning in medical imaging.