Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance

TL;DR

This paper introduces a new neural network model that combines the simplicity of k-winners-take-all with richer dynamics, enabling better encoding of information through iterative excitation-inhibition balance.

Contribution

It presents an intermediate model with explicit inhibitory neurons, improving upon existing models by allowing flexible dynamics and adaptive computations like habituation and clustering.

Findings

The model effectively encodes information into binary cell assemblies.

It demonstrates improved adaptation to input distributions.

A new learning rule for binary weights is proposed.

Abstract

This paper targets the problem of encoding information into binary cell assemblies. Spiking neural networks and k-winners-take-all models are two common approaches, but the first is hard to use for information processing and the second is too simple and lacks important features of the first. We present an intermediate model that shares the computational ease of kWTA and has more flexible and richer dynamics. It uses explicit inhibitory neurons to balance and shape excitation through an iterative procedure. This leads to a recurrent interaction between inhibitory and excitatory neurons that better adapts to the input distribution and performs such computations as habituation, decorrelation, and clustering. To show these, we investigate Hebbian-like learning rules and propose a new learning rule for binary weights with multiple stabilization mechanisms. Our source code is publicly available.