Neural Fourier Filter Bank

TL;DR

The paper introduces a neural Fourier filter bank that decomposes signals into spatial and frequency components using learned wavelet-inspired grids and sine-activated MLPs, achieving efficient, detailed reconstructions.

Contribution

It proposes a novel neural Fourier filter bank that enhances signal decomposition and reconstruction by integrating Fourier features with grid-based spatial decomposition.

Findings

Outperforms state-of-the-art in model compactness

Faster convergence on multiple tasks

Effective in 2D, 3D, and radiance field reconstructions

Abstract

We present a novel method to provide efficient and highly detailed reconstructions. Inspired by wavelets, we learn a neural field that decompose the signal both spatially and frequency-wise. We follow the recent grid-based paradigm for spatial decomposition, but unlike existing work, encourage specific frequencies to be stored in each grid via Fourier features encodings. We then apply a multi-layer perceptron with sine activations, taking these Fourier encoded features in at appropriate layers so that higher-frequency components are accumulated on top of lower-frequency components sequentially, which we sum up to form the final output. We demonstrate that our method outperforms the state of the art regarding model compactness and convergence speed on multiple tasks: 2D image fitting, 3D shape reconstruction, and neural radiance fields. Our code is available at https://github.com/ubc-vision/NFFB.