Structure and Redundancy in Large Language Models: A Spectral Study via Random Matrix Theory

TL;DR

This paper introduces spectral analysis techniques based on Random Matrix Theory to understand, detect failures, and compress large language models, improving reliability and efficiency through novel spectral methods.

Contribution

It presents EigenTrack for real-time detection of model hallucinations and out-of-distribution behavior, and RMT-KD for spectral-based model compression, advancing interpretability and efficiency in large models.

Findings

EigenTrack effectively detects reliability failures early.

RMT-KD produces more compact, energy-efficient models.

Spectral statistics distinguish structured representations from noise.

Abstract

This thesis addresses two persistent and closely related challenges in modern deep learning, reliability and efficiency, through a unified framework grounded in Spectral Geometry and Random Matrix Theory (RMT). As deep networks and large language models continue to scale, their internal behavior becomes increasingly opaque, leading to hallucinations, fragile generalization under distribution shift, and growing computational and energy demands. By analyzing the eigenvalue dynamics of hidden activations across layers and inputs, this work shows that spectral statistics provide a compact, stable, and interpretable lens on model behavior, capable of separating structured, causal representations from noise-dominated variability. Within this framework, the first contribution, EigenTrack, introduces a real-time method for detecting hallucinations and out-of-distribution behavior in large language and vision-language models. EigenTrack transforms streaming activations into spectral descriptors such as entropy, variance, and deviations from the Marchenko-Pastur baseline, and models their temporal evolution using lightweight recurrent classifiers, enabling early detection of reliability failures before they appear in model outputs while offering interpretable insight into representation dynamics. The second contribution, RMT-KD, presents a principled approach to compressing deep networks via random matrix theoretic knowledge distillation. By interpreting outlier eigenvalues in activation spectra as carriers of task-relevant information, RMT-KD progressively projects networks onto lower-dimensional subspaces through iterative self-distillation, yielding significantly more compact and energy-efficient models while preserving accuracy and dense, hardware-friendly structure.