Memory Capacity of a Random Neural Network

TL;DR

This study investigates the information capacity of symmetric random neural networks, revealing that binary weights maximize capacity and increasing quantization levels does not enhance it.

Contribution

It demonstrates that binary weight networks have maximal capacity, challenging assumptions that more quantization levels improve neural network information capacity.

Findings

Binary random neural networks have maximum capacity.

Capacity remains unchanged with increased quantization levels.

Symmetrical matrices are used to analyze network capacity.

Abstract

This paper considers the problem of information capacity of a random neural network. The network is represented by matrices that are square and symmetrical. The matrices have a weight which determines the highest and lowest possible value found in the matrix. The examined matrices are randomly generated and analyzed by a computer program. We find the surprising result that the capacity of the network is a maximum for the binary random neural network and it does not change as the number of quantization levels associated with the weights increases.