A global universality of two-layer neural networks with ReLU activations

TL;DR

This paper proves that two-layer neural networks with ReLU activations are universally dense in function spaces under a suitable norm, demonstrating global convergence properties beyond compact sets.

Contribution

It introduces a new norm enabling the analysis of global convergence, extending universality results to entire function spaces rather than just compact sets.

Findings

Neural networks are dense in function spaces globally.

A new norm facilitates uniform convergence analysis.

Results extend previous compact set convergence studies.

Abstract

In the present study, we investigate a universality of neural networks, which concerns a density of the set of two-layer neural networks in a function spaces. There are many works that handle the convergence over compact sets. In the present paper, we consider a global convergence by introducing a norm suitably, so that our results will be uniform over any compact set.