Concentration inequalities and optimal number of layers for stochastic deep neural networks

TL;DR

This paper establishes concentration inequalities for stochastic deep neural networks, introduces an expected classifier with probabilistic error bounds, and determines the optimal number of layers using an optimal stopping approach.

Contribution

It provides the first concentration inequalities for SDNNs, introduces an expected classifier with error bounds, and identifies the optimal network depth.

Findings

Concentration inequalities for SDNN outputs

Probabilistic bounds for classifier error

Optimal number of layers determined

Abstract

We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.