A multi-scale information geometry reveals the structure of mutual information in neural populations

TL;DR

This paper introduces a multi-scale information geometry based on a Riemannian metric that relates neural population responses to mutual information, providing a new framework for understanding neural coding.

Contribution

It derives a unique, multi-scale Fisher information geometry from first principles that links neural response structure to mutual information, applicable to large neural populations.

Findings

The geometry captures encoding from fine details to global distinctions.

Well encoded stimulus directions are expanded, poorly encoded are contracted.

Eigenvectors of the metric identify features contributing most to information.

Abstract

Understanding how neural population responses represent sensory information is a central problem in systems neuroscience. One approach is to define a representational geometry on stimulus space in which distances reflect how reliably stimuli can be distinguished from neural activity. However, different constructions of these distances can lead to qualitatively different conclusions about the neural code. Here, we show that a unique Riemannian representational geometry emerges from first principles governing how distances contract as stimulus resolution is lost through coarse-graining. This results in a multi-scale extension of the Fisher information metric, capturing encoding structure from fine stimulus details to coarse global distinctions. The resulting geometry is exactly related to the mutual information encoded by the population: well encoded stimulus directions - those contributing more to mutual information - are expanded, whereas poorly encoded directions are contracted. The metric tensor can be estimated using diffusion models, making the framework practical for large neural populations and high-dimensional stimuli. Applied to visual cortical responses to natural images, the eigenvectors of the metric tensor identify stimulus variations that contribute most to information transmission, yielding interpretable features that are robust to modelling choices. Together, these results provide a principled, information-theoretic framework for characterising neural population codes.