Neurons as Detectors of Coherent Sets in Sensory Dynamics

TL;DR

This paper models sensory neurons as self-supervised learners that identify coherent regions in stimulus space using spectral clustering of stochastic dynamical systems, explaining neural responses and functional dichotomies.

Contribution

It introduces a novel framework where neurons detect coherent sets via spectral clustering of the stochastic Koopman operator, extending to nonlinear dynamics and biological plausibility.

Findings

Neurons can detect predictive and retrospective coherent sets.

Spectral clustering of the SKO explains neural temporal filtering.

Framework accounts for neural rectification and functional dichotomies.

Abstract

We model sensory streams as observations from high-dimensional stochastic dynamical systems and conceptualize sensory neurons as self-supervised learners of compact representations of such dynamics. From prior experience, neurons learn coherent sets-regions of stimulus state space whose trajectories evolve cohesively over finite times-and assign membership indices to new stimuli. Coherent sets are identified via spectral clustering of the stochastic Koopman operator (SKO), where the sign pattern of a subdominant singular function partitions the state space into minimally coupled regions. For multivariate Ornstein-Uhlenbeck processes, this singular function reduces to a linear projection onto the dominant singular vector of the whitened state-transition matrix. Encoding this singular vector as a receptive field enables neurons to compute membership indices via the projection sign in a biologically plausible manner. Each neuron detects either a predictive coherent set (stimuli with common futures) or a retrospective coherent set (stimuli with common pasts), suggesting a functional dichotomy among neurons. Since neurons lack access to explicit dynamical equations, the requisite singular vectors must be estimated directly from data, for example, via past-future canonical correlation analysis on lag-vector representations-an approach that naturally extends to nonlinear dynamics. This framework provides a novel account of neuronal temporal filtering, the ubiquity of rectification in neural responses, and known functional dichotomies. Coherent-set clustering thus emerges as a fundamental computation underlying sensory processing and transferable to bio-inspired artificial systems.