Implicit Neural Representations and the Algebra of Complex Wavelets

TL;DR

This paper explores the use of complex wavelets in implicit neural representations (INRs) to better capture high-frequency signal features, proposing new design principles for INR architectures.

Contribution

It introduces the use of complex wavelets in INRs, providing novel design strategies for improved signal representation and high-frequency feature resolution.

Findings

Complex wavelets enhance high-frequency feature capture in INRs.

Decoupling low and band-pass approximations improves signal representation.

Initialization schemes based on signal singularities improve INR performance.

Abstract

Implicit neural representations (INRs) have arisen as useful methods for representing signals on Euclidean domains. By parameterizing an image as a multilayer perceptron (MLP) on Euclidean space, INRs effectively represent signals in a way that couples spatial and spectral features of the signal that is not obvious in the usual discrete representation, paving the way for continuous signal processing and machine learning approaches that were not previously possible. Although INRs using sinusoidal activation functions have been studied in terms of Fourier theory, recent works have shown the advantage of using wavelets instead of sinusoids as activation functions, due to their ability to simultaneously localize in both frequency and space. In this work, we approach such INRs and demonstrate how they resolve high-frequency features of signals from coarse approximations done in the first layer of the MLP. This leads to multiple prescriptions for the design of INR architectures, including the use of complex wavelets, decoupling of low and band-pass approximations, and initialization schemes based on the singularities of the desired signal.