Entropy, mutual information, and systematic measures of structured spiking neural networks

TL;DR

This paper explores entropy, mutual information, and systematic measures in structured spiking neural networks, demonstrating their computability and stability through theoretical proofs and analysis of various network examples.

Contribution

It introduces a coarse-grained approach to compute information-theoretic measures in large neural networks and proves their well-definedness and stability.

Findings

Measures are well-defined and computable for large networks

Coarse-graining captures network rhythm information

Analytical results on properties of information measures

Abstract

The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In order to analyze and compute these information-theoretic measures for large networks, we coarse-grained the data by ignoring the order of spikes that fall into the same small time bin. The resultant coarse-grained entropy mainly capture the information contained in the rhythm produced by a local population of the network. We first proved that these information theoretical measures are well-defined and computable by proving the stochastic stability and the law of large numbers. Then we use three neuronal network examples, from simple to complex, to investigate these information-theoretic measures. Several analytical and computational results about properties of these information-theoretic measures are given.