Approximation capability of two hidden layer feedforward neural networks with fixed weights

TL;DR

This paper demonstrates that a specifically constructed two hidden layer feedforward neural network with fixed weights can approximate any continuous multivariate function with arbitrary precision, outperforming single hidden layer models with fixed weights.

Contribution

It provides a novel construction of a two hidden layer neural network with fixed weights capable of universal approximation for multivariate functions.

Findings

Two hidden layer fixed-weight neural networks can approximate any continuous multivariate function.

The constructed network uses 3d+2 hidden neurons for a d-dimensional input.

This approach shows an advantage over single hidden layer fixed-weight networks.

Abstract

We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the single hidden layer feedforward neural network (SLFN) model, since SLFNs with fixed weights do not have the capability of approximating multivariate functions.