An elementary proof of a universal approximation theorem

TL;DR

This paper provides an elementary proof of a universal approximation theorem for neural networks with three hidden layers, using only undergraduate analysis, making the proof more accessible but with a weaker result.

Contribution

It offers a simpler, more accessible proof of a universal approximation theorem for neural networks with three hidden layers, avoiding advanced mathematical machinery.

Findings

Elementary proof of universal approximation for neural networks

Applicable to networks with three hidden layers and continuous bounded activation functions

Result is weaker than existing theorems but more accessible

Abstract

In this short note, we give an elementary proof of a universal approximation theorem for neural networks with three hidden layers and increasing, continuous, bounded activation function. The result is weaker than the best known results, but the proof is elementary in the sense that no machinery beyond undergraduate analysis is used.