Sparse neural networks with large learning diversity

TL;DR

This paper introduces a simple binary neural network with three levels of sparsity, capable of learning and recalling many messages despite high erasures, enhancing robustness in neural memory models.

Contribution

It presents a novel coded recurrent neural network with three sparsity levels, improving message storage and recall in binary neural systems.

Findings

Able to learn and recall many messages with high erasure resilience

Effective as a classifier and associative memory

Maintains performance despite low connection density

Abstract

Coded recurrent neural networks with three levels of sparsity are introduced. The first level is related to the size of messages, much smaller than the number of available neurons. The second one is provided by a particular coding rule, acting as a local constraint in the neural activity. The third one is a characteristic of the low final connection density of the network after the learning phase. Though the proposed network is very simple since it is based on binary neurons and binary connections, it is able to learn a large number of messages and recall them, even in presence of strong erasures. The performance of the network is assessed as a classifier and as an associative memory.