SL$^{2}$A-INR: Single-Layer Learnable Activation for Implicit Neural Representation

TL;DR

This paper introduces SL$^{2}$A-INR, a hybrid neural network architecture with a learnable single-layer activation, significantly improving the performance of implicit neural representations across various vision tasks.

Contribution

It proposes a novel hybrid INR architecture combining a learnable single-layer activation with traditional ReLU-based MLPs, enhancing high-frequency component capture and signal diversity.

Findings

Sets new benchmarks in accuracy and quality for INR tasks

Improves robustness across diverse vision applications

Achieves superior performance in image and 3D shape tasks

Abstract

Implicit Neural Representation (INR), leveraging a neural network to transform coordinate input into corresponding attributes, has recently driven significant advances in several vision-related domains. However, the performance of INR is heavily influenced by the choice of the nonlinear activation function used in its multilayer perceptron (MLP) architecture. To date, multiple nonlinearities have been investigated, but current INRs still face limitations in capturing high-frequency components and diverse signal types. We show that these challenges can be alleviated by introducing a novel approach in INR architecture. Specifically, we propose SL$^{2}$A-INR, a hybrid network that combines a single-layer learnable activation function with an MLP that uses traditional ReLU activations. Our method performs superior across diverse tasks, including image representation, 3D shape reconstruction, and novel view synthesis. Through comprehensive experiments, SL$^{2}$A-INR sets new benchmarks in accuracy, quality, and robustness for INR. Our Code is publicly available on~\href{https://github.com/Iceage7/SL2A-INR}{\textcolor{magenta}{GitHub}}.