Asymptotic properties of one-layer artificial neural networks with sparse connectivity

TL;DR

This paper establishes a law of large numbers for the parameter distribution in one-layer neural networks with sparse connectivity, as the number of neurons and training iterations grow, providing insights into their asymptotic behavior.

Contribution

It introduces a novel law of large numbers for sparse neural networks, enhancing understanding of their asymptotic properties during training.

Findings

Law of large numbers for parameter distribution

Asymptotic behavior of sparse neural networks

Convergence during stochastic gradient descent

Abstract

A law of large numbers for the empirical distribution of parameters of a one-layer artificial neural networks with sparse connectivity is derived for a simultaneously increasing number of both, neurons and training iterations of the stochastic gradient descent.