An adaptive wavelet-based PINN for problems with localized high-magnitude source

TL;DR

This paper introduces an adaptive wavelet-based PINN that effectively handles localized high-magnitude sources in PDEs, overcoming spectral bias and loss imbalance issues, with theoretical analysis and superior empirical performance.

Contribution

It proposes a novel adaptive wavelet-based PINN framework that dynamically adjusts basis functions, avoids automatic differentiation, and demonstrates theoretical and empirical advantages over existing methods.

Findings

AW-PINN outperforms existing methods on PDEs with localized high-magnitude sources.

The method handles extreme loss imbalances up to 10^{10}:1.

Theoretical analysis shows AW-PINN admits a Gaussian process limit and derives NTK structure.

Abstract

In recent years, physics-informed neural networks (PINNs) have gained significant attention for solving differential equations, although they suffer from two fundamental limitations, namely, spectral bias inherent in neural networks and loss imbalance arising from multiscale phenomena. This paper proposes an adaptive wavelet-based PINN (AW-PINN) to address the extreme loss imbalance characteristic of problems with localized high-magnitude source terms. Such problems frequently arise in various physical applications, such as thermal processing, electro-magnetics, impact mechanics, and fluid dynamics involving localized forcing. The proposed framework dynamically adjusts the wavelet basis function based on residual and supervised loss. This adaptive nature makes AW-PINN handle problems with high-scale features effectively without being memory-intensive. Additionally, AW-PINN does not rely on automatic differentiation to obtain derivatives involved in the loss function, which accelerates the training process. The method operates in two stages, an initial short pre-training phase with fixed bases to select physically relevant wavelet families, followed by an adaptive refinement that adapts scales and translations without populating high-resolution bases across entire domains. Theoretically, we show that under certain assumptions, AW-PINN admits a Gaussian process limit and derive its associated NTK structure. We evaluate AW-PINN on several challenging PDEs featuring localized high-magnitude source terms with extreme loss imbalances having ratios up to $10^{10}:1$. Across these PDEs, including transient heat conduction, highly localized Poisson problems, oscillatory flow equations, and Maxwell equations with a point charge source, AW-PINN consistently outperforms existing methods in its class.