FwNet-ECA: A Classification Model Enhancing Window Attention with Global Receptive Fields via Fourier Filtering Operations

TL;DR

FwNet-ECA introduces a Fourier-based spectral enhancement technique combined with Efficient Channel Attention to improve window attention models, achieving lower computational costs and competitive accuracy in image classification tasks.

Contribution

The paper proposes a novel Fourier transform-based spectral enhancement method integrated with ECA to replace windowed attention, providing a more efficient global receptive field in vision models.

Findings

Achieves lower parameter count and computational overheads compared to shifted window methods.

Maintains competitive accuracy on ImageNet and iCartoonFace datasets.

Filter Enhancement is most effective in shallow layers with larger feature maps.

Abstract

Windowed attention mechanisms were introduced to mitigate the issue of excessive computation inherent in global attention mechanisms. In this paper, we present FwNet-ECA, a novel method that utilizes Fourier transforms paired with learnable weight matrices to enhance the spectral features of images. This method establishes a global receptive field through Filter Enhancement and avoids the use of moving window attention. Additionally, we incorporate the Efficient Channel Attention (ECA) module to improve communication between different channels. Instead of relying on physically shifted windows, our approach leverages frequency domain enhancement to implicitly bridge information across spatial regions. We validate our model on the iCartoonFace dataset and conduct downstream tasks on ImageNet, demonstrating that our model achieves lower parameter counts and computational overheads compared to shifted window approaches, while maintaining competitive accuracy. Furthermore, our visualization operations clearly demonstrated that the Filter Enhancement technique achieves greater effectiveness in the model's shallow layers, where feature maps are relatively larger. This work offers a more efficient and effective alternative for leveraging attention mechanisms in visual processing tasks, alleviating the challenges associated with windowed attention models. Code is available at https://github.com/qingxiaoli/FwNet-ECA