Deep Stable Learning for Out-Of-Distribution Generalization

TL;DR

This paper introduces a deep learning approach that improves out-of-distribution generalization by removing feature dependencies through sample weighting, effectively reducing spurious correlations without relying on domain labels.

Contribution

We propose a novel method that learns sample weights to eliminate feature dependencies, enhancing distribution robustness without requiring domain labels or equal domain capacities.

Findings

Outperforms state-of-the-art methods on multiple benchmarks

Effectively reduces reliance on spurious correlations

Improves generalization across diverse distribution shifts

Abstract

Approaches based on deep neural networks have achieved striking performance when testing data and training data share similar distribution, but can significantly fail otherwise. Therefore, eliminating the impact of distribution shifts between training and testing data is crucial for building performance-promising deep models. Conventional methods assume either the known heterogeneity of training data (e.g. domain labels) or the approximately equal capacities of different domains. In this paper, we consider a more challenging case where neither of the above assumptions holds. We propose to address this problem by removing the dependencies between features via learning weights for training samples, which helps deep models get rid of spurious correlations and, in turn, concentrate more on the true connection between discriminative features and labels. Extensive experiments clearly demonstrate the effectiveness of our method on multiple distribution generalization benchmarks compared with state-of-the-art counterparts. Through extensive experiments on distribution generalization benchmarks including PACS, VLCS, MNIST-M, and NICO, we show the effectiveness of our method compared with state-of-the-art counterparts.