Grounding and Enhancing Grid-based Models for Neural Fields

TL;DR

This paper develops a theoretical framework for grid-based neural models, introduces a new model MulFAGrid with improved generalization, and demonstrates its superior performance in multiple neural field tasks.

Contribution

It provides a systematic analysis of grid-based models via GTK and proposes MulFAGrid, a novel model with better generalization and state-of-the-art results.

Findings

MulFAGrid has a lower generalization bound than previous models.

MulFAGrid achieves state-of-the-art results in 2D and 3D neural field tasks.

The framework enables systematic analysis of grid-based neural models.

Abstract

Many contemporary studies utilize grid-based models for neural field representation, but a systematic analysis of grid-based models is still missing, hindering the improvement of those models. Therefore, this paper introduces a theoretical framework for grid-based models. This framework points out that these models' approximation and generalization behaviors are determined by grid tangent kernels (GTK), which are intrinsic properties of grid-based models. The proposed framework facilitates a consistent and systematic analysis of diverse grid-based models. Furthermore, the introduced framework motivates the development of a novel grid-based model named the Multiplicative Fourier Adaptive Grid (MulFAGrid). The numerical analysis demonstrates that MulFAGrid exhibits a lower generalization bound than its predecessors, indicating its robust generalization performance. Empirical studies reveal that MulFAGrid achieves state-of-the-art performance in various tasks, including 2D image fitting, 3D signed distance field (SDF) reconstruction, and novel view synthesis, demonstrating superior representation ability. The project website is available at https://sites.google.com/view/cvpr24-2034-submission/home.