Escaping Spectral Bias without Backpropagation: Fast Implicit Neural Representations with Extreme Learning Machines

TL;DR

This paper introduces ELM-INR, a novel method for implicit neural representations that avoids backpropagation by using Extreme Learning Machines and domain decomposition, enabling fast, stable, and adaptive high-frequency detail reconstruction.

Contribution

It presents ELM-INR, a backpropagation-free approach utilizing local Extreme Learning Machines for efficient and robust implicit neural representations, along with an adaptive mesh refinement strategy called BEAM.

Findings

ELM-INR achieves fast, stable reconstruction without backpropagation.

The spectral analysis guides adaptive refinement to improve high-frequency detail capture.

BEAM effectively balances spectral complexity across subdomains.

Abstract

Training implicit neural representations (INRs) to capture fine-scale details typically relies on iterative backpropagation and is often hindered by spectral bias when the target exhibits highly non-uniform frequency content. We propose ELM-INR, a backpropagation-free INR that decomposes the domain into overlapping subdomains and fits each local problem using an Extreme Learning Machine (ELM) in closed form, replacing iterative optimization with stable linear least-squares solutions. This design yields fast and numerically robust reconstruction by combining local predictors through a partition of unity. To understand where approximation becomes difficult under fixed local capacity, we analyze the method from a spectral Barron norm perspective, which reveals that global reconstruction error is dominated by regions with high spectral complexity. Building on this insight, we introduce BEAM, an adaptive mesh refinement strategy that balances spectral complexity across subdomains to improve reconstruction quality in capacity-constrained regimes.