Reducing Deep Network Complexity via Sparse Hierarchical Fourier Interaction Networks

TL;DR

This paper introduces Sparse Hierarchical Fourier Interaction Networks, a novel architecture that combines hierarchical Fourier transforms with learnable spectral masking to efficiently model local and global features in visual and linguistic data.

Contribution

It proposes a new neural network architecture that unifies hierarchical Fourier transforms with spectral coefficient masking for improved efficiency and representation.

Findings

Efficiently captures local detail and global context.

Retains only the most informative spectral coefficients.

Leverages natural compressibility of signals.

Abstract

This paper presents a Sparse Hierarchical Fourier Interaction Networks, an architectural building block that unifies three complementary principles of frequency domain modeling: A hierarchical patch wise Fourier transform that affords simultaneous access to local detail and global context; A learnable, differentiable top K masking mechanism which retains only the most informative spectral coefficients, thereby exploiting the natural compressibility of visual and linguistic signals.