Least-Squares ReLU Neural Network (LSNN) Method For Linear Advection-Reaction Equation

TL;DR

This paper introduces a least-squares ReLU neural network method for solving linear advection-reaction equations with discontinuous solutions, automatically capturing interfaces and outperforming traditional mesh-based methods.

Contribution

It proposes a novel neural network approach that efficiently approximates discontinuous solutions and interfaces without mesh, using a least-squares formulation with ReLU networks.

Findings

The method outperforms mesh-based methods in degrees of freedom.

It avoids Gibbs phenomena along discontinuities.

A three-layer ReLU network suffices for 2D discontinuous interfaces.

Abstract

This paper studies least-squares ReLU neural network method for solving the linear advection-reaction problem with discontinuous solution. The method is a discretization of an equivalent least-squares formulation in the set of neural network functions with the ReLU activation function. The method is capable of approximating the discontinuous interface of the underlying problem automatically through the free hyper-planes of the ReLU neural network and, hence, outperforms mesh-based numerical methods in terms of the number of degrees of freedom. Numerical results of some benchmark test problems show that the method can not only approximate the solution with the least number of parameters, but also avoid the common Gibbs phenomena along the discontinuous interface. Moreover, a three-layer ReLU neural network is necessary and sufficient in order to well approximate a discontinuous solution with an interface in $\mathbb{R}^2$ that is not a straight line.