Spectral Concentration at the Edge of Stability: Information Geometry of Kernel Associative Memory

TL;DR

This paper presents a geometric theory explaining the stability and capacity of high-capacity kernel Hopfield networks by analyzing their dynamics on a statistical manifold, revealing a critical boundary linked to the Fisher Information Matrix.

Contribution

It introduces a novel geometric framework connecting spectral concentration, stability, and capacity in kernel Hopfield networks through information geometry and the Edge of Stability concept.

Findings

Identification of the Ridge as the Edge of Stability where Fisher Information becomes singular

Demonstration of dual equilibrium in Riemannian space affecting network dynamics

Unification of learning dynamics and capacity via the Minimum Description Length principle

Abstract

High-capacity kernel Hopfield networks exhibit a \textit{Ridge of Optimization} characterized by extreme stability. While previously linked to \textit{Spectral Concentration}, its origin remains elusive. Here, we analyze the network dynamics on a statistical manifold, revealing that the Ridge corresponds to the Edge of Stability, a critical boundary where the Fisher Information Matrix becomes singular. We demonstrate that the apparent Euclidean force antagonism is a manifestation of \textit{Dual Equilibrium} in the Riemannian space. This unifies learning dynamics and capacity via the Minimum Description Length principle, offering a geometric theory of self-organized criticality.