Dual form Complementary Masking for Domain-Adaptive Image Segmentation

TL;DR

This paper introduces MaskTwins, a novel domain-adaptive image segmentation framework that leverages dual complementary masking and theoretical insights to enhance domain-invariant feature extraction and improve segmentation performance across diverse domains.

Contribution

It provides a theoretical analysis of masked reconstruction as a sparse signal problem and proposes MaskTwins, integrating complementary masks for improved domain generalization in image segmentation.

Findings

MaskTwins outperforms baseline methods in natural image segmentation.

The framework effectively extracts domain-invariant features.

It achieves superior results without separate pre-training.

Abstract

Recent works have correlated Masked Image Modeling (MIM) with consistency regularization in Unsupervised Domain Adaptation (UDA). However, they merely treat masking as a special form of deformation on the input images and neglect the theoretical analysis, which leads to a superficial understanding of masked reconstruction and insufficient exploitation of its potential in enhancing feature extraction and representation learning. In this paper, we reframe masked reconstruction as a sparse signal reconstruction problem and theoretically prove that the dual form of complementary masks possesses superior capabilities in extracting domain-agnostic image features. Based on this compelling insight, we propose MaskTwins, a simple yet effective UDA framework that integrates masked reconstruction directly into the main training pipeline. MaskTwins uncovers intrinsic structural patterns that persist across disparate domains by enforcing consistency between predictions of images masked in complementary ways, enabling domain generalization in an end-to-end manner. Extensive experiments verify the superiority of MaskTwins over baseline methods in natural and biological image segmentation. These results demonstrate the significant advantages of MaskTwins in extracting domain-invariant features without the need for separate pre-training, offering a new paradigm for domain-adaptive segmentation.