Tyler's and Maronna's M-estimators: Non-Asymptotic Concentration Results

TL;DR

This paper derives tight non-asymptotic concentration bounds for Tyler's and Maronna's M-estimators under various distributional assumptions, advancing understanding of their finite-sample behavior in high dimensions.

Contribution

The work provides the first non-asymptotic concentration bounds for these estimators beyond elliptical and Gaussian distributions, using tools from random matrix theory.

Findings

Derived tight concentration bounds for M-estimators

Applicable to sub-Gaussian, log-concave, and convex concentration distributions

Demonstrated utility in sparse covariance and precision matrix estimation

Abstract

Tyler's and Maronna's M-estimators, as well as their regularized variants, are popular robust methods to estimate the scatter or covariance matrix of a multivariate distribution. In this work, we study the non-asymptotic behavior of these estimators, for data sampled from a distribution that satisfies one of the following properties: 1) independent sub-Gaussian entries, up to a linear transformation; 2) log-concave distributions; 3) distributions satisfying a convex concentration property. Our main contribution is the derivation of tight non-asymptotic concentration bounds of these M-estimators around a suitably scaled version of the data sample covariance matrix. Prior to our work, non-asymptotic bounds were derived only for Elliptical and Gaussian distributions. Our proof uses a variety of tools from non asymptotic random matrix theory and high dimensional geometry. Finally, we illustrate the utility of our results on two examples of practical interest: sparse covariance and sparse precision matrix estimation.