The Importance of Background Information for Out of Distribution Generalization

TL;DR

This paper investigates the role of background information in medical image classification for out-of-distribution generalization, proposing a new segmentation mask and emphasizing the importance of training data scale.

Contribution

It introduces a task-specific segmentation mask that covers all relevant regions, improving OOD performance in medical image classification.

Findings

Background regions provide helpful signals for classification.

Using task-specific masks significantly improves OOD test performance.

Scaling up training data enhances generalization beyond ERM.

Abstract

Domain generalization in medical image classification is an important problem for trustworthy machine learning to be deployed in healthcare. We find that existing approaches for domain generalization which utilize ground-truth abnormality segmentations to control feature attributions have poor out-of-distribution (OOD) performance relative to the standard baseline of empirical risk minimization (ERM). We investigate what regions of an image are important for medical image classification and show that parts of the background, that which is not contained in the abnormality segmentation, provides helpful signal. We then develop a new task-specific mask which covers all relevant regions. Utilizing this new segmentation mask significantly improves the performance of the existing methods on the OOD test sets. To obtain better generalization results than ERM, we find it necessary to scale up the training data size in addition to the usage of these task-specific masks.