Information thermodynamics: from physics to neuroscience

TL;DR

This paper explores how principles of information thermodynamics from physics can be applied to understand neural systems, linking energy, information, and learning processes in the brain.

Contribution

It introduces a novel framework combining thermodynamics and information theory to analyze neural inference, learning, and energy costs in neural networks.

Findings

Neural networks can infer probabilistic motion with quantifiable accuracy and energy cost.

Neural systems can learn and store information about particle velocity in synaptic weights.

The framework provides practical tools for studying neural information processing from a physical perspective.

Abstract

This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in neuroscience have been treated separately, in contrast to physics approaches, where the relationship of entropy production with heat is a central idea. It is argued here that also in neural systems information and energy can be considered within the same theoretical framework. Starting from basic ideas of thermodynamics and information theory on a classic Brownian particle, it is shown how noisy neural networks can infer its probabilistic motion. The decoding of the particle motion by neurons is performed with some accuracy and it has some energy cost, and both can be determined using information thermodynamics. In a similar fashion, we also discuss how neural networks in the brain can learn the particle velocity, and maintain that information in the weights of plastic synapses from a physical point of view. Generally, it is shown how the framework of stochastic and information thermodynamics can be used practically to study neural inference, learning, and information storing.