Closed-Form, Provable, and Robust PCA via Leverage Statistics and Innovation Search

TL;DR

This paper introduces a new robust PCA method that leverages the connection between Innovation Values and Leverage Scores, providing theoretical guarantees and demonstrating superior performance and robustness in noisy settings.

Contribution

The paper establishes a novel link between Innovation Values and Leverage Scores, leading to a fast, closed-form robust PCA algorithm with provable guarantees.

Findings

The proposed method outperforms existing algorithms in experiments.

It is robust against noise and various outlier distributions.

The approach is computationally efficient and theoretically sound.

Abstract

The idea of Innovation Search, which was initially proposed for data clustering, was recently used for outlier detection. In the application of Innovation Search for outlier detection, the directions of innovation were utilized to measure the innovation of the data points. We study the Innovation Values computed by the Innovation Search algorithm under a quadratic cost function and it is proved that Innovation Values with the new cost function are equivalent to Leverage Scores. This interesting connection is utilized to establish several theoretical guarantees for a Leverage Score based robust PCA method and to design a new robust PCA method. The theoretical results include performance guarantees with different models for the distribution of outliers and the distribution of inliers. In addition, we demonstrate the robustness of the algorithms against the presence of noise. The numerical and theoretical studies indicate that while the presented approach is fast and closed-form, it can outperform most of the existing algorithms.