Asymptotic convexity of wide and shallow neural networks

TL;DR

This paper demonstrates that the input-output map of wide, shallow neural networks approximates a convex function, providing a theoretical explanation for their strong performance.

Contribution

It introduces a theoretical framework showing the asymptotic convexity of wide, shallow neural networks' input-output maps.

Findings

Epigraph of the network's input-output map approximates a convex function

Provides a plausible explanation for the good performance of wide, shallow networks

Establishes a connection between network architecture and convexity properties

Abstract

For a simple model of shallow and wide neural networks, we show that the epigraph of its input-output map as a function of the network parameters approximates epigraph of a. convex function in a precise sense. This leads to a plausible explanation of their observed good performance.