T-Rex: Fitting a Robust Factor Model via Expectation-Maximization

TL;DR

This paper introduces T-Rex, a robust EM algorithm for fitting low-rank plus diagonal covariance models using Tyler’s M-estimator, effectively handling heavy tails and outliers in high-dimensional data.

Contribution

It develops a novel EM algorithm based on Tyler's M-estimator to robustly fit factor models with low-rank plus diagonal structure, improving resilience to outliers.

Findings

Demonstrates robustness in synthetic data experiments.

Shows improved direction-of-arrival estimation in nonuniform noise.

Achieves accurate subspace recovery with real data.

Abstract

Over the past decades, there has been a surge of interest in studying low-dimensional structures within high-dimensional data. Statistical factor models $-$ i.e., low-rank plus diagonal covariance structures $-$ offer a powerful framework for modeling such structures. However, traditional methods for fitting statistical factor models, such as principal component analysis (PCA) or maximum likelihood estimation assuming the data is Gaussian, are highly sensitive to heavy tails and outliers in the observed data. In this paper, we propose a novel expectation-maximization (EM) algorithm for robustly fitting statistical factor models. Our approach is based on Tyler's M-estimator of the scatter matrix for an elliptical distribution, and consists of solving Tyler's maximum likelihood estimation problem while imposing a structural constraint that enforces the low-rank plus diagonal covariance structure. We present numerical experiments on both synthetic and real examples, demonstrating the robustness of our method for direction-of-arrival estimation in nonuniform noise and subspace recovery.