Summary of Information Theoretic Quantities

TL;DR

This paper introduces key information theoretic quantities like entropy and mutual information, emphasizing their utility in analyzing neural data and information flow in the nervous system.

Contribution

It provides clear definitions of fundamental information theoretic measures and discusses their practical estimation and application in neuroscience research.

Findings

Information theoretic measures are versatile for neural data analysis.

These quantities allow comparison across different experiments.

They enable analysis of neural responses in single trials.

Abstract

Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. As a framework it has a number of useful properties: it provides a general measure sensitive to any relationship, not only linear effects; its quantities have meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single experimental trials, rather than in averages over multiple trials. A variety of information theoretic quantities are in common use in neuroscience - including the Shannon entropy, Kullback-Leibler divergence, and mutual information. In this entry, we introduce and define these quantities. Further details on how these quantities can be estimated in practice are provided in the entry "Estimation of Information-Theoretic Quantities" and examples of application of these techniques in neuroscience can be found in the entry "Applications of Information-Theoretic Quantities in Neuroscience".