Generalization bounds for deep convolutional neural networks

TL;DR

This paper derives generalization error bounds for deep convolutional neural networks that depend on training loss, parameters, and weight distance, but not on input size, supported by experiments on CIFAR-10.

Contribution

It introduces input-size independent generalization bounds for convolutional networks and validates them through empirical experiments on CIFAR-10.

Findings

Bounds are independent of input pixel count.

Experimental results support the theoretical bounds.

Generalization gaps are explained by the proposed bounds.

Abstract

We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They are independent of the number of pixels in the input, and the height and width of hidden feature maps. We present experiments using CIFAR-10 with varying hyperparameters of a deep convolutional network, comparing our bounds with practical generalization gaps.