Convergence and Fluctuations of Regularized Tyler Estimators

TL;DR

This paper analyzes the asymptotic behavior of regularized Tyler estimators (RTEs) for scatter matrices, establishing their convergence and fluctuation properties in the regime where the number of observations grows large with fixed dimension.

Contribution

It provides the first analysis of RTEs' convergence and fluctuations when the sample size increases with fixed data dimension, complementing existing large-dimensional results.

Findings

RTEs converge to a deterministic matrix as sample size increases with fixed dimension.

Fluctuations of RTEs are asymptotically Gaussian with a covariance depending on the population covariance and regularization parameter.

Results help in understanding the statistical properties of RTEs for better regularization parameter selection.

Abstract

This article studies the behavior of regularized Tyler estimators (RTEs) of scatter matrices. The key advantages of these estimators are twofold. First, they guarantee by construction a good conditioning of the estimate and second, being a derivative of robust Tyler estimators, they inherit their robustness properties, notably their resilience to the presence of outliers. Nevertheless, one major problem that poses the use of RTEs in practice is represented by the question of setting the regularization parameter $\rho$. While a high value of $\rho$ is likely to push all the eigenvalues away from zero, it comes at the cost of a larger bias with respect to the population covariance matrix. A deep understanding of the statistics of RTEs is essential to come up with appropriate choices for the regularization parameter. This is not an easy task and might be out of reach, unless one considers asymptotic regimes wherein the number of observations $n$ and/or their size $N$ increase together. First asymptotic results have recently been obtained under the assumption that $N$ and $n$ are large and commensurable. Interestingly, no results concerning the regime of $n$ going to infinity with $N$ fixed exist, even though the investigation of this assumption has usually predated the analysis of the most difficult $N$ and $n$ large case. This motivates our work. In particular, we prove in the present paper that the RTEs converge to a deterministic matrix when $n\to\infty$ with $N$ fixed, which is expressed as a function of the theoretical covariance matrix. We also derive the fluctuations of the RTEs around this deterministic matrix and establish that these fluctuations converge in distribution to a multivariate Gaussian distribution with zero mean and a covariance depending on the population covariance and the parameter $\rho$.