Bits from Biology for Computational Intelligence

TL;DR

This paper explores how information theory can analyze neural systems to uncover algorithms and representations, guiding the development of biologically inspired computing systems through information-theoretic measures and local information dynamics.

Contribution

It introduces methods for analyzing neural information encoding and processing, and discusses recent advances in information-theoretic analysis of neural data and complex systems.

Findings

Information-theoretic analysis reveals neural algorithms and representations.

New methods quantify information about the environment in neural systems.

Framework of local information dynamics decomposes neural information processing.

Abstract

Computational intelligence is broadly defined as biologically-inspired computing. Usually, inspiration is drawn from neural systems. This article shows how to analyze neural systems using information theory to obtain constraints that help identify the algorithms run by such systems and the information they represent. Algorithms and representations identified information-theoretically may then guide the design of biologically inspired computing systems (BICS). The material covered includes the necessary introduction to information theory and the estimation of information theoretic quantities from neural data. We then show how to analyze the information encoded in a system about its environment, and also discuss recent methodological developments on the question of how much information each agent carries about the environment either uniquely, or redundantly or synergistically together with others. Last, we introduce the framework of local information dynamics, where information processing is decomposed into component processes of information storage, transfer, and modification -- locally in space and time. We close by discussing example applications of these measures to neural data and other complex systems.