Estimation consistante de l'architecture des perceptrons multicouches

TL;DR

This paper proves that using an information criterion allows for consistent estimation of the true number of hidden units in multilayer perceptrons with Gaussian noise, addressing overestimation issues.

Contribution

It introduces a method to consistently estimate the number of hidden units in MLPs by minimizing a suitable information criterion under certain conditions.

Findings

Consistent estimation of hidden units achieved

Information matrix invertibility issues addressed

Method applicable with parameters in a compact set

Abstract

We consider regression models involving multilayer perceptrons (MLP) with one hidden layer and a Gaussian noise. The estimation of the parameters of the MLP can be done by maximizing the likelihood of the model. In this framework, it is difficult to determine the true number of hidden units because the information matrix of Fisher is not invertible if this number is overestimated. However, if the parameters of the MLP are in a compact set, we prove that the minimization of a suitable information criteria leads to consistent estimation of the true number of hidden units.