On the approximation by single hidden layer feedforward neural networks with fixed weights

TL;DR

This paper proves that single hidden layer feedforward neural networks with fixed weights can approximate any continuous univariate function with just two neurons, but cannot do so for multivariate functions.

Contribution

It provides a constructive proof that fixed-weight SLFNs with two neurons can approximate all univariate continuous functions, highlighting limitations for multivariate cases.

Findings

Two-neuron fixed-weight SLFNs can approximate any univariate continuous function.

Numerical examples demonstrate the approximation capability.

Fixed weights limit the approximation of multivariate functions.

Abstract

Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the number of neurons in the hidden layer. The more this number, the more the probability of the considered network to give precise results. In this note, we constructively prove that SLFNs with the fixed weight $1$ and two neurons in the hidden layer can approximate any continuous function on a compact subset of the real line. The applicability of this result is demonstrated in various numerical examples. Finally, we show that SLFNs with fixed weights cannot approximate all continuous multivariate functions.