Efficient Estimation of Multidimensional Regression Model using Multilayer Perceptrons

TL;DR

This paper introduces a novel cost function for estimating multidimensional nonlinear regression models with multilayer perceptrons, achieving asymptotic optimality without prior noise covariance knowledge.

Contribution

It proposes using the log-determinant of the empirical error covariance matrix as a cost function, simplifying estimation and model selection in MLP-based regression.

Findings

Cost function leads to asymptotically optimal estimators

Simplifies testing the number of parameters in MLPs

Numerical experiments confirm theoretical advantages

Abstract

This work concerns the estimation of multidimensional nonlinear regression models using multilayer perceptrons (MLPs). The main problem with such models is that we need to know the covariance matrix of the noise to get an optimal estimator. However, we show in this paper that if we choose as the cost function the logarithm of the determinant of the empirical error covariance matrix, then we get an asymptotically optimal estimator. Moreover, under suitable assumptions, we show that this cost function leads to a very simple asymptotic law for testing the number of parameters of an identifiable MLP. Numerical experiments confirm the theoretical results.