Embedding of Low-Dimensional Sensory Dynamics in Recurrent Networks: Implications for the Geometry of Neural Representation

TL;DR

This paper models recurrent neural networks driven by low-dimensional sensory inputs, showing how such networks develop smooth manifolds representing sensory dynamics and linking their geometry to predictive performance.

Contribution

It introduces a theoretical framework combining synchronization and delay-embedding to explain the emergence and geometry of sensory manifolds in recurrent networks.

Findings

Recurrent networks develop smooth manifolds embedding sensory dynamics.

State separation correlates with prediction accuracy and thresholds.

Numerical experiments confirm manifold shapes and the role of network size.

Abstract

Neural population activity in sensory cortex is organized on low-dimensional manifolds, but why such manifolds arise and what determines their geometry remain unclear. We model cortical populations as recurrent circuits driven by low-dimensional regular sensory dynamics (circles, tori). Combining generalized synchronization and delay-embedding theory, we show that contracting recurrent networks generically develop smooth internal manifolds embedding the sensory dynamics. The dimensional requirement is modest: N>2d suffices, where d is the intrinsic sensory dimension (compatible with Whitney and Takens bounds). We prove a prediction-separation result linking representational geometry to predictive performance without assuming contraction: accurate prediction forces state separation up to a resolution set by prediction error, yielding categorical boundaries, metameric equivalence, and discrimination thresholds. Numerical experiments with trained tanh RNNs recover ring- and torus-shaped hidden manifolds; state separation improves sharply at the 2d+1 threshold. Training pushes networks beyond strict contraction, yet embedding persists, indicating sufficient but not necessary conditions. These results provide a mechanistic account of why sensory manifolds emerge in recurrent circuits and how prediction constrains their resolution.