Asymptotic behavior of eigenvalues of large rank perturbations of large random matrices

TL;DR

This paper analyzes the asymptotic eigenvalue behavior of large deformed Wigner matrices, relevant for understanding neural network weight spectra and pruning techniques.

Contribution

It develops an asymptotic analysis for full rank perturbations with increasing outlier eigenvalues in large random matrices.

Findings

Provides a theoretical framework for eigenvalue distribution in deformed Wigner matrices.

Connects spectral properties to neural network pruning methods.

Analyzes the impact of outlier eigenvalues on matrix spectra.

Abstract

The paper is concerned with deformed Wigner random matrices. These matrices are closely related to Deep Neural Networks (DNNs): weight matrices of trained DNNs could be represented in the form $R + S$, where $R$ is random and $S$ is highly correlated. The spectrum of such matrices plays a key role in rigorous underpinning of the novel pruning technique based on Random Matrix Theory. In practice, the spectrum of the matrix $S$ can be rather complicated. In this paper, we develop an asymptotic analysis for the case of full rank $S$ with increasing number of outlier eigenvalues.