A simple normalization technique using window statistics to improve the out-of-distribution generalization on medical images

TL;DR

This paper introduces window normalization (WIN), a simple technique that perturbs feature statistics with local window-based statistics to enhance CNN generalization on heterogeneous medical images, especially for out-of-distribution data.

Contribution

The paper proposes WIN, a novel normalization method using window statistics, and WIN-WIN, a self-distillation approach, to improve OOD generalization in medical image classification.

Findings

WIN improves OOD generalization across multiple tasks and datasets.

WIN-WIN enhances model robustness with minimal additional computation.

Experimental results show significant performance gains over existing normalization methods.

Abstract

Since data scarcity and data heterogeneity are prevailing for medical images, well-trained Convolutional Neural Networks (CNNs) using previous normalization methods may perform poorly when deployed to a new site. However, a reliable model for real-world clinical applications should be able to generalize well both on in-distribution (IND) and out-of-distribution (OOD) data (e.g., the new site data). In this study, we present a novel normalization technique called window normalization (WIN) to improve the model generalization on heterogeneous medical images, which is a simple yet effective alternative to existing normalization methods. Specifically, WIN perturbs the normalizing statistics with the local statistics computed on the window of features. This feature-level augmentation technique regularizes the models well and improves their OOD generalization significantly. Taking its advantage, we propose a novel self-distillation method called WIN-WIN for classification tasks. WIN-WIN is easily implemented with twice forward passes and a consistency constraint, which can be a simple extension for existing methods. Extensive experimental results on various tasks (6 tasks) and datasets (24 datasets) demonstrate the generality and effectiveness of our methods.