Contrastive Image Synthesis and Self-supervised Feature Adaptation for Cross-Modality Biomedical Image Segmentation

TL;DR

This paper introduces CISFA, a novel framework for cross-modality biomedical image segmentation that combines contrastive image synthesis with self-supervised feature adaptation, improving segmentation accuracy across CT and MRI images.

Contribution

The paper proposes a one-sided generative model with a weighted patch-wise contrastive loss and contrastive training of the encoder, offering a new explicit method for domain-independent feature learning.

Findings

Outperforms state-of-the-art domain adaptation methods in segmentation accuracy.

Generates synthetic images with less organ shape distortion.

Effective on CT and MRI abdominal and cardiac datasets.

Abstract

This work presents a novel framework CISFA (Contrastive Image synthesis and Self-supervised Feature Adaptation)that builds on image domain translation and unsupervised feature adaptation for cross-modality biomedical image segmentation. Different from existing works, we use a one-sided generative model and add a weighted patch-wise contrastive loss between sampled patches of the input image and the corresponding synthetic image, which serves as shape constraints. Moreover, we notice that the generated images and input images share similar structural information but are in different modalities. As such, we enforce contrastive losses on the generated images and the input images to train the encoder of a segmentation model to minimize the discrepancy between paired images in the learned embedding space. Compared with existing works that rely on adversarial learning for feature adaptation, such a method enables the encoder to learn domain-independent features in a more explicit way. We extensively evaluate our methods on segmentation tasks containing CT and MRI images for abdominal cavities and whole hearts. Experimental results show that the proposed framework not only outputs synthetic images with less distortion of organ shapes, but also outperforms state-of-the-art domain adaptation methods by a large margin.