Encoding Semantic Priors into the Weights of Implicit Neural Representation

TL;DR

This paper introduces SPW, a method that encodes semantic priors into the weights of implicit neural representations, significantly improving their performance across multiple vision tasks without additional inference costs.

Contribution

The paper proposes a novel reparameterization technique, SPW, that embeds semantic information into INR weights, enhancing their capacity and efficiency in visual signal representation.

Findings

SPW improves INR performance on image fitting, CT, MRI, and view synthesis tasks.

Models with SPW have lower weight redundancy and learn more novel features.

SPW enhances semantic understanding in implicit neural representations.

Abstract

Implicit neural representation (INR) has recently emerged as a promising paradigm for signal representations, which takes coordinates as inputs and generates corresponding signal values. Since these coordinates contain no semantic features, INR fails to take any semantic information into consideration. However, semantic information has been proven critical in many vision tasks, especially for visual signal representation. This paper proposes a reparameterization method termed as SPW, which encodes the semantic priors to the weights of INR, thus making INR contain semantic information implicitly and enhancing its representational capacity. Specifically, SPW uses the Semantic Neural Network (SNN) to extract both low- and high-level semantic information of the target visual signal and generates the semantic vector, which is input into the Weight Generation Network (WGN) to generate the weights of INR model. Finally, INR uses the generated weights with semantic priors to map the coordinates to the signal values. After training, we only retain the generated weights while abandoning both SNN and WGN, thus SPW introduces no extra costs in inference. Experimental results show that SPW can improve the performance of various INR models significantly on various tasks, including image fitting, CT reconstruction, MRI reconstruction, and novel view synthesis. Further experiments illustrate that model with SPW has lower weight redundancy and learns more novel representations, validating the effectiveness of SPW.