Rank Is Not Capacity: Spectral Occupancy for Latent Graph Models

TL;DR

This paper introduces Spectra, a spectral analysis method for latent graph models that replaces fixed rank hyperparameters with a spectrum-based measure of capacity, enabling better control and understanding of model behavior.

Contribution

It proposes a spectral framework that quantifies and controls model capacity through eigenvalue distributions, moving beyond traditional fixed-rank hyperparameters.

Findings

Spectra effectively traces performance-capacity trade-offs across various networks.

It performs competitively with strong link-prediction baselines.

Spectra provides a principled, model-based measure of capacity in overparameterized regimes.

Abstract

Graph representation learning has become a standard approach for analyzing networked data, with latent embeddings widely used for link prediction, community detection, and related tasks. Yet a basic design choice, the latent dimension, is still treated as a brittle hyperparameter, fixed before training and tuned by held-out performance. Learned factors are also identifiable only up to rotation and rescaling, so the nominal rank rarely coincides with the quantity that governs model behavior. We propose Spectral Prefix Extraction and Capacity-Targeted Representation Analysis (Spectra), which replaces rank as the unit of analysis with the spectrum of a learned positive semidefinite kernel, trace-normalized so that spectra are comparable across fits. The normalized eigenvalues form a distribution on the simplex, and their Shannon effective rank acts both as a summary of learned capacity and as a controllable training-time coordinate: a single scalar shapes this realized dimension during training, and bisection targets any desired value within the rank cap. To theoretically support that, we show local regularity and monotonicity of the realized-dimension profile. Across collaboration, social, biological, and infrastructure networks, Spectra traces performance--capacity frontiers that make the trade-off between predictive accuracy and realized dimension visible. It performs competitively with strong link-prediction baselines, yields aligned lower-capacity views of the same fitted model through spectral prefixes, and provides a principled handle on capacity in the overparameterized regime. Capacity thus becomes a property of the fitted model rather than a hyperparameter of the training.