Risk bounds for aggregated shallow neural networks using Gaussian prior

TL;DR

This paper provides sharp, nonasymptotic risk bounds for aggregated shallow neural networks with Gaussian priors, combining estimation and approximation errors to achieve minimax rates over Sobolev classes.

Contribution

It offers a precise nonasymptotic analysis of estimation error in PAC-Bayesian bounds for aggregated neural networks, advancing theoretical understanding.

Findings

Derived sharp risk bounds for neural network aggregation

Established minimax rates over Sobolev classes

Reviewed bounds on neural network approximation errors

Abstract

Analysing statistical properties of neural networks is a central topic in statistics and machine learning. However, most results in the literature focus on the properties of the neural network minimizing the training error. The goal of this paper is to consider aggregated neural networks using a Gaussian prior. The departure point of our approach is an arbitrary aggregate satisfying the PAC-Bayesian inequality. The main contribution is a precise nonasymptotic assessment of the estimation error appearing in the PAC-Bayes bound. We also review available bounds on the error of approximating a function by a neural network. Combining bounds on estimation and approximation errors, we establish risk bounds that are sharp enough to lead to minimax rates of estimation over Sobolev smoothness classes.