Nonlinear classification of neural manifolds with contextual information

TL;DR

This paper introduces a theoretical framework for analyzing how neural representations, influenced by contextual information, can be classified nonlinearly, extending existing linear capacity measures to better understand neural and artificial systems.

Contribution

It develops an exact formula for context-dependent manifold capacity that accounts for geometry and context correlations, enabling analysis of nonlinear neural representations.

Findings

Validated on synthetic and real data.

Captured early-stage deep network reformatting.

Enhanced understanding of context-dependent neural computation.

Abstract

Understanding how neural systems efficiently process information through distributed representations is a fundamental challenge at the interface of neuroscience and machine learning. Recent approaches analyze the statistical and geometrical attributes of neural representations as population-level mechanistic descriptors of task implementation. In particular, manifold capacity has emerged as a promising framework linking population geometry to the separability of neural manifolds. However, this metric has been limited to linear readouts. To address this limitation, we introduce a theoretical framework that leverages latent directions in input space, which can be related to contextual information. We derive an exact formula for the context-dependent manifold capacity that depends on manifold geometry and context correlations, and validate it on synthetic and real data. Our framework's increased expressivity captures representation reformatting in deep networks at early stages of the layer hierarchy, previously inaccessible to analysis. As context-dependent nonlinearity is ubiquitous in neural systems, our data-driven and theoretically grounded approach promises to elucidate context-dependent computation across scales, datasets, and models.