Neural Networks in Numerical Analysis and Approximation Theory

TL;DR

This thesis explores the approximation power of Neural Networks for solving elliptic PDEs and their relation to classical approximation spaces, demonstrating their effectiveness and theoretical properties.

Contribution

It introduces a neural network-based Galerkin method for elliptic PDEs and characterizes the approximation space of neural networks in relation to Besov spaces.

Findings

Neural networks can approximate the inverse of positive-definite matrices.

The neural network approximation space relates to Besov spaces, with smoother functions being approximated faster.

A neural network-based Galerkin method effectively solves elliptic PDEs.

Abstract

In this Master Thesis, we study the approximation capabilities of Neural Networks in the context of numerical resolution of elliptic PDEs and Approximation Theory. First of all, in Chapter 1, we introduce the mathematical definition of Neural Networks and perform some basic estimates on their composition and parallelization. Then, we implement in Chapter 2 the Galerkin method using Neural Network. In particular, we manage to build a Neural Network that approximates the inverse of positive-definite symmetric matrices, which allows to get a Garlerkin numerical solution of elliptic PDEs. Finally, in Chapter 3, we introduce the approximation space of Neural Networks, a space which consists of functions in $L^p$ that are approximated at a certain rate when increasing the number of weights of Neural Networks. We find the relation of this space with the Besov space: the smoother a function is, the faster it can be approximated with Neural Networks when increasing the number of weights.