Representation Geometry as a Diagnostic for Out-of-Distribution Robustness

TL;DR

This paper introduces a geometry-based diagnostic framework that analyzes learned embeddings' structure to predict out-of-distribution robustness, enabling label-free, interpretable assessment and checkpoint selection.

Contribution

It proposes novel geometric invariants derived from class-conditional graphs to evaluate OOD robustness without labels, validated across multiple models and benchmarks.

Findings

Lower spectral complexity correlates with higher OOD accuracy.

Higher mean curvature indicates better robustness.

Representation geometry signals reflect meaningful structure, not superficial statistics.

Abstract

Robust generalization under distribution shift remains difficult to monitor and optimize in the absence of target-domain labels, as models with similar in-distribution accuracy can exhibit markedly different out-of-distribution (OOD) performance. While prior work has focused on training-time regularization and low-order representation statistics, little is known about whether the geometric structure of learned embeddings provides reliable post-hoc signals of robustness. We propose a geometry-based diagnostic framework that constructs class-conditional mutual k-nearest-neighbor graphs from in-distribution embeddings and extracts two complementary invariants: a global spectral complexity proxy based on the reduced log-determinant of the normalized Laplacian, and a local smoothness measure based on Ollivier--Ricci curvature. Across multiple architectures, training regimes, and corruption benchmarks, we find that lower spectral complexity and higher mean curvature consistently predict stronger OOD accuracy across checkpoints. Controlled perturbations and topological analyses further show that these signals reflect meaningful representation structure rather than superficial embedding statistics. Our results demonstrate that representation geometry enables interpretable, label-free robustness diagnosis and supports reliable unsupervised checkpoint selection under distribution shift.