STAF: Sinusoidal Trainable Activation Functions for Implicit Neural Representation

TL;DR

STAF introduces trainable sinusoidal activation functions that adaptively learn spectral components, significantly enhancing the accuracy and efficiency of implicit neural representations across various signal and inverse problem tasks.

Contribution

The paper proposes STAF, a novel trainable sinusoidal activation function that improves spectral learning and representation in neural networks, addressing limitations of spectral bias in INRs.

Findings

STAF outperforms state-of-the-art methods in signal reconstruction tasks.

It accelerates convergence and enhances expressivity in neural representations.

Effective across diverse applications like shape, image, audio, and NeRF.

Abstract

Implicit Neural Representations (INRs) have emerged as a powerful framework for modeling continuous signals. The spectral bias of ReLU-based networks is a well-established limitation, restricting their ability to capture fine-grained details in target signals. While previous works have attempted to mitigate this issue through frequency-based encodings or architectural modifications, these approaches often introduce additional complexity and do not fully address the underlying challenge of learning high-frequency components efficiently. We introduce Sinusoidal Trainable Activation Functions (STAF), designed to directly tackle this limitation by enabling networks to adaptively learn and represent complex signals with higher precision and efficiency. STAF inherently modulates its frequency components, allowing for self-adaptive spectral learning. This capability significantly improves convergence speed and expressivity, making STAF highly effective for both signal representations and inverse problems. Through extensive evaluations across a range of tasks, including signal representation (shape, image, audio) and inverse problems (super-resolution, denoising), as well as neural radiance fields (NeRF), we demonstrate that STAF consistently outperforms state-of-the-art methods in accuracy and reconstruction fidelity. These results establish STAF as a robust solution to spectral bias and the capacity--convergence tradeoff, with broad applicability in computer vision and graphics. Our codebase is publicly accessible at https://github.com/AlirezaMorsali/STAF.