Computing the degrees of freedom of rank-regularized estimators and cousins

TL;DR

This paper develops a framework using Stein's Unbiased Risk Estimate to accurately compute degrees of freedom for spectral regularized low-rank matrix estimators, aiding in tuning parameter selection.

Contribution

It generalizes degrees of freedom estimation to a broad class of spectral estimators, extending previous methods and introducing new techniques for non-differentiable cases.

Findings

Derived degrees of freedom estimates for spectral estimators

Verified applicability of Stein's lemma in this context

Introduced Gaussian convolution techniques for non-differentiable estimators

Abstract

Estimating a low rank matrix from its linear measurements is a problem of central importance in contemporary statistical analysis. The choice of tuning parameters for estimators remains an important challenge from a theoretical and practical perspective. To this end, Stein's Unbiased Risk Estimate (SURE) framework provides a well-grounded statistical framework for degrees of freedom estimation. In this paper, we use the SURE framework to obtain degrees of freedom estimates for a general class of spectral regularized matrix estimators, generalizing beyond the class of estimators that have been studied thus far. To this end, we use a result due to Shapiro (2002) pertaining to the differentiability of symmetric matrix valued functions, developed in the context of semidefinite optimization algorithms. We rigorously verify the applicability of Stein's lemma towards the derivation of degrees of freedom estimates; and also present new techniques based on Gaussian convolution to estimate the degrees of freedom of a class of spectral estimators to which Stein's lemma is not directly applicable.