FAMLP: A Frequency-Aware MLP-Like Architecture For Domain Generalization

TL;DR

This paper introduces FAMLP, a frequency-aware MLP-like architecture designed to improve domain generalization by filtering domain-specific features in the frequency domain, achieving state-of-the-art results.

Contribution

It proposes the first MLP-like backbone with frequency filtering and low-rank enhancement for improved domain generalization performance.

Findings

Outperforms state-of-the-art methods by 3-9% on three benchmarks.

Introduces an adaptive Fourier filter layer with learnable parameters.

Employs a momentum update strategy for stabilized training.

Abstract

MLP-like models built entirely upon multi-layer perceptrons have recently been revisited, exhibiting the comparable performance with transformers. It is one of most promising architectures due to the excellent trade-off between network capability and efficiency in the large-scale recognition tasks. However, its generalization performance to heterogeneous tasks is inferior to other architectures (e.g., CNNs and transformers) due to the extensive retention of domain information. To address this problem, we propose a novel frequency-aware MLP architecture, in which the domain-specific features are filtered out in the transformed frequency domain, augmenting the invariant descriptor for label prediction. Specifically, we design an adaptive Fourier filter layer, in which a learnable frequency filter is utilized to adjust the amplitude distribution by optimizing both the real and imaginary parts. A low-rank enhancement module is further proposed to rectify the filtered features by adding the low-frequency components from SVD decomposition. Finally, a momentum update strategy is utilized to stabilize the optimization to fluctuation of model parameters and inputs by the output distillation with weighted historical states. To our best knowledge, we are the first to propose a MLP-like backbone for domain generalization. Extensive experiments on three benchmarks demonstrate significant generalization performance, outperforming the state-of-the-art methods by a margin of 3%, 4% and 9%, respectively.