Recurrent infomax generates cell assemblies, avalanches, and simple cell-like selectivity

TL;DR

This paper introduces recurrent infomax (RI), a learning algorithm for recurrent neural networks that maximizes information retention, leading to the emergence of cell-like selectivity, spontaneous activity, and neuronal avalanches similar to biological neural systems.

Contribution

The paper proposes a novel infomax-based learning algorithm for recurrent networks that explains complex neural phenomena and stimulus selectivity.

Findings

Recurrent infomax induces Gabor-like selectivity in visual cortex models.

Networks exhibit spontaneous cell assembly and synfire chain activity.

Neuronal avalanches emerge without external input.

Abstract

Through evolution, animals have acquired central nervous systems (CNSs), which are extremely efficient information processing devices that improve an animal's adaptability to various environments. It has been proposed that the process of information maximization (infomax), which maximizes the information transmission from the input to the output of a feedforward network, may provide an explanation of the stimulus selectivity of neurons in CNSs. However, CNSs contain not only feedforward but also recurrent synaptic connections, and little is known about information retention over time in such recurrent networks. Here, we propose a learning algorithm based on infomax in a recurrent network, which we call "recurrent infomax" (RI). RI maximizes information retention and thereby minimizes information loss in a network. We find that feeding in external inputs consisting of information obtained from photographs of natural scenes into an RI-based model of a recurrent network results in the appearance of Gabor-like selectivity quite similar tothat existing in simple cells of the primary visual cortex (V1). More importantly, we find that without external input, this network exhibits cell assembly-like and synfire chain-like spontaneous activity and a critical neuronal avalanche. RI provides a simple framework to explain a wide range of phenomena observed in in vivo and in vitro neuronal networks, and it should provide a novel understanding of experimental results for multineuronal activity and plasticity from an information-theoretic point of view.