Shallow neural network yields regularization for ill-posed inverse problems

TL;DR

This paper introduces a neural network-based regularization method for solving nonlinear ill-posed inverse problems, demonstrating its theoretical properties and robustness through numerical examples.

Contribution

It establishes universal approximation theorems for neural networks in ill-posed problems and introduces the expanding neural network method as a novel iterative regularization scheme.

Findings

Small networks suffice for high noise data to ensure stability.

Larger networks risk overfitting and instability.

Convergence rates are derived under standard regularization assumptions.

Abstract

In this paper, we establish universal approximation theorems for neural networks applied to general nonlinear ill-posed operator equations. In addition to the approximation error, the measurement error is also taken into account in our error estimation. We introduce the expanding neural network method as a novel iterative regularization scheme and prove its regularization properties under different a priori assumptions about the exact solutions. Within this framework, the number of neurons serves as both the regularization parameter and iteration number. We demonstrate that for data with high noise levels, a small network architecture is sufficient to obtain a stable solution, whereas a larger architecture may compromise stability due to overfitting. Furthermore, under standard assumptions in regularization theory, we derive convergence rate results for neural networks in the context of variational regularization. Several numerical examples are presented to illustrate the robustness of the proposed neural network-based algorithms.