Split-Layer: Enhancing Implicit Neural Representation by Maximizing the Dimensionality of Feature Space

TL;DR

This paper introduces the split-layer, a novel MLP reformulation that enhances the representational capacity of implicit neural representations by expanding feature space dimensionality efficiently, leading to improved performance across various inverse signal reconstruction tasks.

Contribution

The split-layer method significantly increases INR capacity by dividing layers into parallel branches and combining outputs via Hadamard product, avoiding high computational costs.

Findings

Outperforms existing INR methods in 2D image fitting

Achieves superior results in 3D shape representation

Enhances 5D view synthesis quality

Abstract

Implicit neural representation (INR) models signals as continuous functions using neural networks, offering efficient and differentiable optimization for inverse problems across diverse disciplines. However, the representational capacity of INR defined by the range of functions the neural network can characterize, is inherently limited by the low-dimensional feature space in conventional multilayer perceptron (MLP) architectures. While widening the MLP can linearly increase feature space dimensionality, it also leads to a quadratic growth in computational and memory costs. To address this limitation, we propose the split-layer, a novel reformulation of MLP construction. The split-layer divides each layer into multiple parallel branches and integrates their outputs via Hadamard product, effectively constructing a high-degree polynomial space. This approach significantly enhances INR's representational capacity by expanding the feature space dimensionality without incurring prohibitive computational overhead. Extensive experiments demonstrate that the split-layer substantially improves INR performance, surpassing existing methods across multiple tasks, including 2D image fitting, 2D CT reconstruction, 3D shape representation, and 5D novel view synthesis.