Improved Implicit Neural Representation with Fourier Reparameterized Training

TL;DR

This paper introduces a Fourier reparameterization technique for implicit neural representations that alleviates spectral bias in MLPs, leading to more detailed and artifact-free reconstructions across various tasks.

Contribution

The paper proposes a novel Fourier reparameterization method that theoretically and practically reduces spectral bias in MLP-based INRs, improving their accuracy and texture quality.

Findings

Fourier reparameterization improves approximation accuracy.

Method enhances texture richness and reduces artifacts.

Validated across multiple MLP architectures.

Abstract

Implicit Neural Representation (INR) as a mighty representation paradigm has achieved success in various computer vision tasks recently. Due to the low-frequency bias issue of vanilla multi-layer perceptron (MLP), existing methods have investigated advanced techniques, such as positional encoding and periodic activation function, to improve the accuracy of INR. In this paper, we connect the network training bias with the reparameterization technique and theoretically prove that weight reparameterization could provide us a chance to alleviate the spectral bias of MLP. Based on our theoretical analysis, we propose a Fourier reparameterization method which learns coefficient matrix of fixed Fourier bases to compose the weights of MLP. We evaluate the proposed Fourier reparameterization method on different INR tasks with various MLP architectures, including vanilla MLP, MLP with positional encoding and MLP with advanced activation function, etc. The superiority approximation results on different MLP architectures clearly validate the advantage of our proposed method. Armed with our Fourier reparameterization method, better INR with more textures and less artifacts can be learned from the training data.