FUTON: Fourier Tensor Network for Implicit Neural Representations

TL;DR

FUTON introduces a Fourier tensor network model for implicit neural representations that combines Fourier basis functions with low-rank tensor decomposition, achieving faster training and better generalization than traditional MLP-based INRs.

Contribution

The paper proposes FUTON, a novel Fourier tensor network that models signals with spectral and low-rank structures, providing theoretical guarantees and improved empirical performance.

Findings

Outperforms state-of-the-art MLP INRs in image and volume tasks.

Trains 2-5 times faster than existing methods.

Achieves better generalization in inverse problems like denoising and super-resolution.

Abstract

Implicit neural representations (INRs) have emerged as powerful tools for encoding signals, yet dominant MLP-based designs often suffer from slow convergence, overfitting to noise, and poor extrapolation. We introduce FUTON (Fourier Tensor Network), which models signals as generalized Fourier series whose coefficients are parameterized by a low-rank tensor decomposition. FUTON implicitly expresses signals as weighted combinations of orthonormal, separable basis functions, combining complementary inductive biases: Fourier bases capture smoothness and periodicity, while the low-rank parameterization enforces low-dimensional spectral structure. We provide theoretical guarantees through a universal approximation theorem and derive an inference algorithm with complexity linear in the spectral resolution and the input dimension. On image and volume representation, FUTON consistently outperforms state-of-the-art MLP-based INRs while training 2--5$\times$ faster. On inverse problems such as image denoising and super-resolution, FUTON generalizes better and converges faster.