MaxStyle: Adversarial Style Composition for Robust Medical Image Segmentation

TL;DR

MaxStyle is a novel data augmentation framework that enhances the robustness of medical image segmentation models against unseen domain shifts by adversarially maximizing style diversity using a single training dataset.

Contribution

It introduces an auxiliary style-augmented decoder and adversarial style search to improve out-of-domain robustness without multi-domain data.

Findings

Significantly improves robustness against unseen corruptions.

Enhances performance across multiple unseen sites and image sequences.

Effective in low- and high-data regimes.

Abstract

Convolutional neural networks (CNNs) have achieved remarkable segmentation accuracy on benchmark datasets where training and test sets are from the same domain, yet their performance can degrade significantly on unseen domains, which hinders the deployment of CNNs in many clinical scenarios. Most existing works improve model out-of-domain (OOD) robustness by collecting multi-domain datasets for training, which is expensive and may not always be feasible due to privacy and logistical issues. In this work, we focus on improving model robustness using a single-domain dataset only. We propose a novel data augmentation framework called MaxStyle, which maximizes the effectiveness of style augmentation for model OOD performance. It attaches an auxiliary style-augmented image decoder to a segmentation network for robust feature learning and data augmentation. Importantly, MaxStyle augments data with improved image style diversity and hardness, by expanding the style space with noise and searching for the worst-case style composition of latent features via adversarial training. With extensive experiments on multiple public cardiac and prostate MR datasets, we demonstrate that MaxStyle leads to significantly improved out-of-distribution robustness against unseen corruptions as well as common distribution shifts across multiple, different, unseen sites and unknown image sequences under both low- and high-training data settings. The code can be found at https://github.com/cherise215/MaxStyle.