Diffusion Operator Geometry of Feedforward Representations

TL;DR

This paper introduces a smooth operator-theoretic framework using Gaussian diffusion operators to analyze the geometry of neural network representations, providing stable and interpretable observables.

Contribution

It develops a novel operator-based approach for studying neural representations, offering closed-form class affinities and stability analysis under perturbations.

Findings

Closed-form class affinities and spectra for Gaussian class-conditional models.

Operator observables vary smoothly with feature perturbations.

MNIST experiments show observables track training and stability.

Abstract

Neural networks transform data through learned representations whose geometry affects separation, contraction, and generalization. Recent work studies this geometry using discrete curvature on neighborhood graphs, suggesting Ricci-flow-like behavior across layers. We develop a smooth operator-theoretic alternative for feedforward representation snapshots. Each feature cloud induces a Gaussian-kernel diffusion Markov operator, and transport, spectral, label-boundary, and local-scale observables are derived from this single object via Bakry-Emery $\Gamma$-calculus. In a balanced Gaussian class-conditional snapshot model with shared covariance, the population operator has closed-form class affinities, leakage, and coarse spectra, all controlled by pairwise regularized Mahalanobis separations $c_\varepsilon^{(a,b)}$. We also prove that the resulting operator observables vary smoothly under feature perturbations, while hard neighborhood-graph diagnostics can change discontinuously. Synthetic experiments validate the closed-form Gaussian bridge, while learned MNIST experiments show that the same operator observables track training, width, and perturbation stability. Together, these results give a stable operator-geometric framework for analyzing feedforward representation geometry.