A posteriori analysis of neural network approximations

TL;DR

This paper develops a posteriori error estimates for neural network approximations of solutions to PDEs, enabling error monitoring and control during optimization, with validation through numerical experiments.

Contribution

It introduces a novel error estimation framework applicable to neural network solutions, extending finite element analysis techniques to machine learning approximations.

Findings

Error estimates are equivalent to the sum of two residuals.

Estimators can be reliably computed and used for error control.

Numerical experiments validate the effectiveness of the proposed methods.

Abstract

In a general setting, we study a posteriori estimates used in finite element analysis to measure the error between a solution and its approximation. The latter is not necessarily generated by a finite element method. We show that the error is equivalent to the sum of two residuals provided that the underlying variational formulation is well posed. The first contribution is the projection of the residual to a finite-dimensional space and is therefore computable, while the second one can be reliably estimated by a computable upper bound in many practical scenarios. Assuming sufficiently accurate quadrature, our findings can be used to estimate the error of, e.g., neural network outputs. Two important applications can be considered during optimization: first, the estimators are used to monitor the error in each solver step, or, second, the two estimators are included in the loss functional, and therefore provide control over the error. As a model problem, we consider a second-order elliptic partial differential equation and discuss different variational formulations thereof, including several options to include boundary conditions in the estimators. Various numerical experiments are presented to validate our findings.