On quadrature rules for solving Partial Differential Equations using Neural Networks

TL;DR

This paper examines quadrature challenges in neural network-based PDE solvers and proposes multiple methods like Monte Carlo, adaptive integration, polynomial approximation, and regularization to address them, analyzing their pros and cons.

Contribution

It introduces and compares various quadrature strategies tailored for neural network PDE solutions, highlighting their applicability based on problem dimensionality.

Findings

Monte Carlo methods are recommended for high-dimensional problems.

Adaptive integration and polynomial approximations are suitable for low-dimensional cases.

Regularization offers a dimension-independent approach but requires regularity assumptions.

Abstract

Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may arise in these applications and propose different alternatives to overcome them, namely: Monte Carlo methods, adaptive integration, polynomial approximations of the Neural Network output, and the inclusion of regularization terms in the loss. We also discuss the advantages and limitations of each proposed alternative. We advocate the use of Monte Carlo methods for high dimensions (above 3 or 4), and adaptive integration or polynomial approximations for low dimensions (3 or below). The use of regularization terms is a mathematically elegant alternative that is valid for any spacial dimension, however, it requires certain regularity assumptions on the solution and complex mathematical analysis when dealing with sophisticated Neural Networks.