Efficient Shallow Ritz Method For 1D Diffusion-Reaction Problems

TL;DR

This paper introduces a damped block Newton method for the shallow Ritz approach, improving approximation order for 1D diffusion-reaction problems using neural networks, with efficient computational cost and practical mesh optimization.

Contribution

It presents a novel damped block Newton method tailored for shallow Ritz neural networks, addressing challenges in diffusion-reaction problems and achieving near-optimal approximation with linear cost.

Findings

The dBN method achieves nearly optimal approximation order.

The method effectively handles dense, ill-conditioned matrices.

Numerical experiments demonstrate mesh point optimization.

Abstract

This paper studies the shallow Ritz method for solving one-dimensional diffusion-reaction problems. The method is capable of improving the order of approximation for non-smooth problems. By following a similar approach to the one presented in [9], we present a damped block Newton (dBN) method to achieve nearly optimal order of approximation. The dBN method optimizes the Ritz functional by alternating between the linear and non-linear parameters of the shallow ReLU neural network (NN). For diffusion-reaction problems, new difficulties arise: (1) for the linear parameters, the mass matrix is dense and even more ill-conditioned than the stiffness matrix, and (2) for the non-linear parameters, the Hessian matrix is dense and may be singular. This paper addresses these challenges, resulting in a dBN method with computational cost of ${\cal O}(n)$. The ideas presented for diffusion-reaction problems can also be applied to least-squares approximation problems. For both applications, starting with the non-linear parameters as a uniform partition, numerical experiments show that the dBN method moves the mesh points to nearly optimal locations.