Multi-level Neural Networks for Accurate Solutions of Boundary-Value Problems

TL;DR

This paper introduces a multi-level neural network approach that iteratively refines solutions to boundary-value problems, significantly improving accuracy and error control over traditional deep learning methods.

Contribution

A novel multi-level neural network methodology that iteratively reduces residual errors to achieve highly accurate solutions for PDE boundary-value problems.

Findings

Error can be reduced to machine precision in some cases

Method captures smaller solution scales at each level

Effective in 1D and 2D numerical examples

Abstract

The solution to partial differential equations using deep learning approaches has shown promising results for several classes of initial and boundary-value problems. However, their ability to surpass, particularly in terms of accuracy, classical discretization methods such as the finite element methods, remains a significant challenge. Deep learning methods usually struggle to reliably decrease the error in their approximate solution. A new methodology to better control the error for deep learning methods is presented here. The main idea consists in computing an initial approximation to the problem using a simple neural network and in estimating, in an iterative manner, a correction by solving the problem for the residual error with a new network of increasing complexity. This sequential reduction of the residual of the partial differential equation allows one to decrease the solution error, which, in some cases, can be reduced to machine precision. The underlying explanation is that the method is able to capture at each level smaller scales of the solution using a new network. Numerical examples in 1D and 2D are presented to demonstrate the effectiveness of the proposed approach. This approach applies not only to physics informed neural networks but to other neural network solvers based on weak or strong formulations of the residual.