Robust Principal Component Analysis: A Median of Means Approach

TL;DR

This paper introduces MoMPCA, a robust PCA method based on the Median of Means principle, which effectively handles outliers and achieves optimal convergence without strong assumptions.

Contribution

It proposes a novel MoM-based PCA approach that is computationally efficient, robust to outliers, and theoretically optimal under minimal assumptions.

Findings

Achieves optimal convergence rates in high-dimensional settings

Robust to outliers without requiring assumptions on their distribution

Validated through simulations and real data applications

Abstract

Principal Component Analysis (PCA) is a fundamental tool for data visualization, denoising, and dimensionality reduction. It is widely popular in Statistics, Machine Learning, Computer Vision, and related fields. However, PCA is well-known to fall prey to outliers and often fails to detect the true underlying low-dimensional structure within the dataset. Following the Median of Means (MoM) philosophy, recent supervised learning methods have shown great success in dealing with outlying observations without much compromise to their large sample theoretical properties. This paper proposes a PCA procedure based on the MoM principle. Called the \textbf{M}edian of \textbf{M}eans \textbf{P}rincipal \textbf{C}omponent \textbf{A}nalysis (MoMPCA), the proposed method is not only computationally appealing but also achieves optimal convergence rates under minimal assumptions. In particular, we explore the non-asymptotic error bounds of the obtained solution via the aid of the Rademacher complexities while granting absolutely no assumption on the outlying observations. The derived concentration results are not dependent on the dimension because the analysis is conducted in a separable Hilbert space, and the results only depend on the fourth moment of the underlying distribution in the corresponding norm. The proposal's efficacy is also thoroughly showcased through simulations and real data applications.