Decompose, Adjust, Compose: Effective Normalization by Playing with Frequency for Domain Generalization

TL;DR

This paper introduces a frequency domain normalization technique, PCNorm and its variants, to improve domain generalization in computer vision by better separating style and content, leading to state-of-the-art results.

Contribution

It proposes a novel spectral decomposition-based normalization method, PCNorm, and variants CCNorm and SCNorm, to enhance domain-agnostic feature learning for robust vision models.

Findings

DAC-SC achieves 65.6% average accuracy on five datasets.

Proposed methods outperform recent domain generalization techniques.

Spectral normalization effectively separates style and content in frequency domain.

Abstract

Domain generalization (DG) is a principal task to evaluate the robustness of computer vision models. Many previous studies have used normalization for DG. In normalization, statistics and normalized features are regarded as style and content, respectively. However, it has a content variation problem when removing style because the boundary between content and style is unclear. This study addresses this problem from the frequency domain perspective, where amplitude and phase are considered as style and content, respectively. First, we verify the quantitative phase variation of normalization through the mathematical derivation of the Fourier transform formula. Then, based on this, we propose a novel normalization method, PCNorm, which eliminates style only as the preserving content through spectral decomposition. Furthermore, we propose advanced PCNorm variants, CCNorm and SCNorm, which adjust the degrees of variations in content and style, respectively. Thus, they can learn domain-agnostic representations for DG. With the normalization methods, we propose ResNet-variant models, DAC-P and DAC-SC, which are robust to the domain gap. The proposed models outperform other recent DG methods. The DAC-SC achieves an average state-of-the-art performance of 65.6% on five datasets: PACS, VLCS, Office-Home, DomainNet, and TerraIncognita.