Phase Matching for Out-of-Distribution Generalization

TL;DR

This paper introduces PhaMa, a phase matching method leveraging Fourier phase components to improve out-of-distribution generalization in deep neural networks, achieving state-of-the-art results.

Contribution

It proposes a novel phase matching approach, PhaMa, that manipulates Fourier phase information and establishes spatial relationships for enhanced domain generalization.

Findings

PhaMa outperforms existing methods on multiple benchmarks.

Perturbing amplitude spectra improves robustness.

Fourier-based SCM clarifies DG relationships.

Abstract

The Fourier transform, an explicit decomposition method for visual signals, has been employed to explain the out-of-distribution generalization behaviors of Deep Neural Networks (DNNs). Previous studies indicate that the amplitude spectrum is susceptible to the disturbance caused by distribution shifts, whereas the phase spectrum preserves highly-structured spatial information that is crucial for robust visual representation learning. Inspired by this insight, this paper is dedicated to clarifying the relationships between Domain Generalization (DG) and the frequency components. Specifically, we provide distribution analysis and empirical experiments for the frequency components. Based on these observations, we propose a Phase Matching approach, termed PhaMa, to address DG problems. To this end, PhaMa introduces perturbations on the amplitude spectrum and establishes spatial relationships to match the phase components with patch contrastive learning. Experiments on multiple benchmarks demonstrate that our proposed method achieves state-of-the-art performance in domain generalization and out-of-distribution robustness tasks. Beyond vanilla analysis and experiments, we further clarify the relationships between the Fourier components and DG problems by introducing a Fourier-based Structural Causal Model (SCM).