On the capacity of neural networks

TL;DR

This thesis investigates and compares the capacity of various neural network models, from classical perceptrons to quantum neural nets, using combinatorial and statistical mechanical methods.

Contribution

It introduces a unified approach to analyze neural network capacity, extending classical results to quantum models and removing dependency on training rules.

Findings

Perceptron capacity analyzed with combinatorial methods

Statistical mechanics applied to associative memory capacity

Extension of capacity analysis to quantum neural networks

Abstract

The aim of this thesis is to compare the capacity of different models of neural networks. We start by analysing the problem solving capacity of a single perceptron using a simple combinatorial argument. After some observations on the storage capacity of a basic network, known as an associative memory, we introduce a powerful statistical mechanical approach to calculate its capacity in the training rule-dependent Hopfield model. With the aim of finding a more general definition that can be applied even to quantum neural nets, we then follow Gardner's work, which let us get rid of the dependency on the training rule, and comment the results obtained by Lewenstein et al. by applying Gardner's methods on a recently proposed quantum perceptron model.