Synergistic Integration of Coordinate Network and Tensorial Feature for Improving Neural Radiance Fields from Sparse Inputs

TL;DR

This paper introduces a novel method combining multi-plane representation with coordinate-based MLPs to enhance neural radiance fields from sparse data, improving detail capture and training efficiency.

Contribution

It proposes a synergistic integration of multi-plane and coordinate-based networks with residual connections and progressive training for better sparse-input NeRFs.

Findings

Outperforms baseline models on static and dynamic NeRFs with sparse inputs.

Achieves comparable results with fewer parameters.

Accelerates training through progressive feature disentanglement.

Abstract

The multi-plane representation has been highlighted for its fast training and inference across static and dynamic neural radiance fields. This approach constructs relevant features via projection onto learnable grids and interpolating adjacent vertices. However, it has limitations in capturing low-frequency details and tends to overuse parameters for low-frequency features due to its bias toward fine details, despite its multi-resolution concept. This phenomenon leads to instability and inefficiency when training poses are sparse. In this work, we propose a method that synergistically integrates multi-plane representation with a coordinate-based MLP network known for strong bias toward low-frequency signals. The coordinate-based network is responsible for capturing low-frequency details, while the multi-plane representation focuses on capturing fine-grained details. We demonstrate that using residual connections between them seamlessly preserves their own inherent properties. Additionally, the proposed progressive training scheme accelerates the disentanglement of these two features. We demonstrate empirically that our proposed method not only outperforms baseline models for both static and dynamic NeRFs with sparse inputs, but also achieves comparable results with fewer parameters.