Deep Learning-based Schemes for Singularly Perturbed Convection-Diffusion Problems

TL;DR

This paper critically evaluates deep learning-based numerical schemes, especially PINNs, for solving singularly perturbed convection-diffusion PDEs with low-regularity solutions, highlighting their strengths and limitations.

Contribution

It provides an extensive numerical comparison of PINNs and traditional methods for low-regularity PDE solutions, focusing on singularly perturbed convection-diffusion problems.

Findings

PINNs face challenges with low-regularity solutions due to optimization issues.

Performance of PINNs degrades as multiscale parameters approach zero.

Traditional numerical schemes may outperform PINNs in certain singularly perturbed cases.

Abstract

Deep learning-based numerical schemes such as Physically Informed Neural Networks (PINNs) have recently emerged as an alternative to classical numerical schemes for solving Partial Differential Equations (PDEs). They are very appealing at first sight because implementing vanilla versions of PINNs based on strong residual forms is easy, and neural networks offer very high approximation capabilities. However, when the PDE solutions are low regular, an expert insight is required to build deep learning formulations that do not incur in variational crimes. Optimization solvers are also significantly challenged, and can potentially spoil the final quality of the approximated solution due to the convergence to bad local minima, and bad generalization capabilities. In this paper, we present an exhaustive numerical study of the merits and limitations of these schemes when solutions exhibit low-regularity, and compare performance with respect to more benign cases when solutions are very smooth. As a support for our study, we consider singularly perturbed convection-diffusion problems where the regularity of solutions typically degrades as certain multiscale parameters go to zero.