A new approach to generalisation error of machine learning algorithms: Estimates and convergence

TL;DR

This paper introduces a novel approach to estimating the generalisation error of deep neural networks, providing error bounds and convergence results without structural assumptions on the networks, under mild regularity conditions on the target function.

Contribution

It offers a new method for error estimation and convergence analysis in neural network regression, independent of network architecture specifics.

Findings

Error estimates without structural assumptions

Convergence of neural network approximations

Applicability under mild regularity conditions

Abstract

In this work we consider a model problem of deep neural learning, namely the learning of a given function when it is assumed that we have access to its point values on a finite set of points. The deep neural network interpolant is the the resulting approximation of f, which is obtained by a typical machine learning algorithm involving a given DNN architecture and an optimisation step, which is assumed to be solved exactly. These are among the simplest regression algorithms based on neural networks. In this work we introduce a new approach to the estimation of the (generalisation) error and to convergence. Our results include (i) estimates of the error without any structural assumption on the neural networks and under mild regularity assumptions on the learning function f (ii) convergence of the approximations to the target function f by only requiring that the neural network spaces have appropriate approximation capability.