RUNNs: Ritz-Uzawa Neural Networks for Solving Variational Problems

TL;DR

RUNNs introduce an iterative neural network framework leveraging Ritz and Uzawa methods to effectively solve challenging PDEs, including those with low regularity and high oscillations.

Contribution

The paper presents RUNNs, a novel iterative neural network approach combining Ritz minimization and Uzawa loops to improve PDE solving stability and accuracy.

Findings

RUNNs accurately resolve highly oscillatory solutions.

They recover discontinuous solutions from distributional sources.

The method reduces bias and variance in specific formulations.

Abstract

Solving Partial Differential Equations (PDEs) using neural networks presents different challenges, including integration errors and spectral bias, often leading to poor approximations. In addition, standard neural network-based methods, such as Physics-Informed Neural Networks (PINNs), often lack stability when dealing with PDEs characterized by low-regularity solutions. To address these limitations, we introduce the Ritz--Uzawa Neural Networks (RUNNs) framework, an iterative methodology to solve strong, weak, and ultra-weak variational formulations. Rewriting the PDE as a sequence of Ritz-type minimization problems within a Uzawa loop provides an iterative framework that, in specific cases, reduces both bias and variance during training. We demonstrate that the strong formulation offers a passive variance reduction mechanism, whereas variance remains persistent in weak and ultra-weak regimes. Furthermore, we address the spectral bias of standard architectures through a data-driven frequency tuning strategy. By initializing a Sinusoidal Fourier Feature Mapping based on the Normalized Cumulative Power Spectral Density (NCPSD) of previous residuals or their proxies, the network dynamically adapts its bandwidth to capture high-frequency components and severe singularities. Numerical experiments demonstrate the robustness of RUNNs, accurately resolving highly oscillatory solutions and successfully recovering a discontinuous $L^2$ solution from a distributional $H^{-2}$ source -- a scenario where standard energy-based methods fail.