Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals

TL;DR

Conv-INR introduces a convolution-based implicit neural representation that effectively captures local structures and high-frequency details in multimodal visual signals, outperforming traditional MLP-based INRs across various tasks.

Contribution

This work presents the first INR model based entirely on convolution, addressing key limitations of MLP-based INRs in representing high-frequency and local signal features.

Findings

Conv-INR outperforms MLP-based INRs in image fitting, medical reconstruction, and view synthesis.

Conv-INR effectively captures high-frequency components and local structures.

Reparameterization methods further improve Conv-INR performance without extra inference cost.

Abstract

Implicit neural representation (INR) has recently emerged as a promising paradigm for signal representations. Typically, INR is parameterized by a multiplayer perceptron (MLP) which takes the coordinates as the inputs and generates corresponding attributes of a signal. However, MLP-based INRs face two critical issues: i) individually considering each coordinate while ignoring the connections; ii) suffering from the spectral bias thus failing to learn high-frequency components. While target visual signals usually exhibit strong local structures and neighborhood dependencies, and high-frequency components are significant in these signals, the issues harm the representational capacity of INRs. This paper proposes Conv-INR, the first INR model fully based on convolution. Due to the inherent attributes of convolution, Conv-INR can simultaneously consider adjacent coordinates and learn high-frequency components effectively. Compared to existing MLP-based INRs, Conv-INR has better representational capacity and trainability without requiring primary function expansion. We conduct extensive experiments on four tasks, including image fitting, CT/MRI reconstruction, and novel view synthesis, Conv-INR all significantly surpasses existing MLP-based INRs, validating the effectiveness. Finally, we raise three reparameterization methods that can further enhance the performance of the vanilla Conv-INR without introducing any extra inference cost.