Fast and Robust: Computationally Efficient Covariance Estimation for Sub-Weibull Vectors

TL;DR

This paper introduces a computationally efficient covariance estimator for high-dimensional Sub-Weibull vectors that maintains spectral properties and achieves near-optimal error bounds, suitable for heavy-tailed data.

Contribution

It proposes a novel Cross-Fitted Norm-Truncated Estimator that is computationally efficient and theoretically optimal for Sub-Weibull distributions, improving robustness and scalability.

Findings

Achieves near-optimal sub-Gaussian rate of convergence.

Requires only $O(Nd^2)$ operations, matching the theoretical lower bound.

Effectively handles heavy-tailed data with exponential tail decay.

Abstract

High-dimensional covariance estimation is notoriously sensitive to outliers. While statistically optimal estimators exist for general heavy-tailed distributions, they often rely on computationally expensive techniques like semidefinite programming or iterative M-estimation ($O(d^3)$). In this work, we target the specific regime of \textbf{Sub-Weibull distributions} (characterized by stretched exponential tails $\exp(-t^\alpha)$). We investigate a computationally efficient alternative: the \textbf{Cross-Fitted Norm-Truncated Estimator}. Unlike element-wise truncation, our approach preserves the spectral geometry while requiring $O(Nd^2)$ operations, which represents the theoretical lower bound for constructing a full covariance matrix. Although spherical truncation is geometrically suboptimal for anisotropic data, we prove that within the Sub-Weibull class, the exponential tail decay compensates for this mismatch. Leveraging weighted Hanson-Wright inequalities, we derive non-asymptotic error bounds showing that our estimator recovers the optimal sub-Gaussian rate $\tilde{O}(\sqrt{r(\Sigma)/N})$ with high probability. This provides a scalable solution for high-dimensional data that exhibits tails heavier than Gaussian but lighter than polynomial decay.