Spectral Identifiability for Interpretable Probe Geometry

TL;DR

This paper introduces the Spectral Identifiability Principle (SIP), a spectral condition that predicts the stability of linear probes in neural network interpretability, linking eigengap size, sample size, and reliability.

Contribution

It formalizes a spectral mechanism for probe stability, providing a verifiable diagnostic to anticipate unreliable probes before downstream distortion.

Findings

Spectral eigengap determines probe stability.

Closing eigengap causes phase-transition instability.

Spectral inspection predicts probe reliability.

Abstract

Linear probes are widely used to interpret and evaluate neural representations, yet their reliability remains unclear, as probes may appear accurate in some regimes but collapse unpredictably in others. We uncover a spectral mechanism behind this phenomenon and formalize it as the Spectral Identifiability Principle (SIP), a verifiable Fisher-inspired condition for probe stability. When the eigengap separating task-relevant directions is larger than the Fisher estimation error, the estimated subspace concentrates and accuracy remains consistent, whereas closing this gap induces instability in a phase-transition manner. Our analysis connects eigengap geometry, sample size, and misclassification risk through finite-sample reasoning, providing an interpretable diagnostic rather than a loose generalization bound. Controlled synthetic studies, where Fisher quantities are computed exactly, confirm these predictions and show how spectral inspection can anticipate unreliable probes before they distort downstream evaluation.