Cauchy noise loss for stochastic optimization of random matrix models via free deterministic equivalents

TL;DR

This paper introduces a novel spectral distribution-based parameter estimation method for random matrix models using Cauchy noise, applicable even with a single sample, and includes a new dimensionality recovery technique.

Contribution

It proposes a new optimization approach leveraging free probability and Cauchy noise, enabling parameter estimation and rank recovery with limited data.

Findings

Effective parameter estimation with single sample matrices.

Successful rank recovery even for large true ranks.

Demonstrated robustness of the method through experiments.

Abstract

For random matrix models, the parameter estimation based on the traditional likelihood functions is not straightforward in particular when we have only one sample matrix. We introduce a new parameter optimization method for random matrix models which works even in such a case. The method is based on the spectral distribution instead of the traditional likelihood. In the method, the Cauchy noise has an essential role because the free deterministic equivalent, which is a tool in free probability theory, allows us to approximate the spectral distribution perturbed by Cauchy noises by a smooth and accessible density function. Moreover, we study an asymptotic property of determination gap, which has a similar role as generalization gap. Besides, we propose a new dimensionality recovery method for the signal-plus-noise model, and experimentally demonstrate that it recovers the rank of the signal part even if the true rank is not small. It is a simultaneous rank selection and parameter estimation procedure.