M\"untz-Sz\'asz Networks: Neural Architectures with Learnable Power-Law Bases

TL;DR

M"untz-Sz"asz Networks (MSN) introduce learnable fractional power bases in neural architectures, significantly improving approximation of singular functions and physics-informed problems over traditional fixed-activation networks.

Contribution

The paper presents MSN, a novel neural architecture with learnable fractional power bases, grounded in approximation theory, enabling superior approximation of singular functions and physics problems.

Findings

MSN achieves lower error with fewer parameters compared to MLPs.

MSN attains approximation rates of $ ext{O}(| ext{exponent} - ext{target}|^2)$ for singular functions.

On physics-informed benchmarks, MSN outperforms standard networks by 3-6 times.

Abstract

Standard neural network architectures employ fixed activation functions (ReLU, tanh, sigmoid) that are poorly suited for approximating functions with singular or fractional power behavior, a structure that arises ubiquitously in physics, including boundary layers, fracture mechanics, and corner singularities. We introduce M\"untz-Sz\'asz Networks (MSN), a novel architecture that replaces fixed smooth activations with learnable fractional power bases grounded in classical approximation theory. Each MSN edge computes $\phi(x) = \sum_k a_k |x|^{\mu_k} + \sum_k b_k \mathrm{sign}(x)|x|^{\lambda_k}$, where the exponents $\{\mu_k, \lambda_k\}$ are learned alongside the coefficients. We prove that MSN inherits universal approximation from the M\"untz-Sz\'asz theorem and establish novel approximation rates: for functions of the form $|x|^\alpha$, MSN achieves error $\mathcal{O}(|\mu - \alpha|^2)$ with a single learned exponent, whereas standard MLPs require $\mathcal{O}(\epsilon^{-1/\alpha})$ neurons for comparable accuracy. On supervised regression with singular target functions, MSN achieves 5-8x lower error than MLPs with 10x fewer parameters. Physics-informed neural networks (PINNs) represent a particularly demanding application for singular function approximation; on PINN benchmarks including a singular ODE and stiff boundary-layer problems, MSN achieves 3-6x improvement while learning interpretable exponents that match the known solution structure. Our results demonstrate that theory-guided architectural design can yield dramatic improvements for scientifically-motivated function classes.