Robust low-rank covariance matrix estimation with a general pattern of missing values

TL;DR

This paper introduces a robust low-rank covariance matrix estimation method that effectively handles incomplete data with arbitrary missing patterns, combining robustness to heavy tails and low-rank structure for improved real-world data analysis.

Contribution

It proposes a novel covariance estimation procedure based on a robust low-rank model using EM algorithm, capable of managing general missing data patterns.

Findings

Validated on simulated datasets showing accurate estimation.

Improved classification and clustering performance on real multispectral data.

Effective handling of arbitrary missing data patterns.

Abstract

This paper tackles the problem of robust covariance matrix estimation when the data is incomplete. Classical statistical estimation methodologies are usually built upon the Gaussian assumption, whereas existing robust estimation ones assume unstructured signal models. The former can be inaccurate in real-world data sets in which heterogeneity causes heavy-tail distributions, while the latter does not profit from the usual low-rank structure of the signal. Taking advantage of both worlds, a covariance matrix estimation procedure is designed on a robust (mixture of scaled Gaussian) low-rank model by leveraging the observed-data likelihood function within an expectation-maximization algorithm. It is also designed to handle general pattern of missing values. The proposed procedure is first validated on simulated data sets. Then, its interest for classification and clustering applications is assessed on two real data sets with missing values, which include multispectral and hyperspectral time series.