A Linearly Convergent Algorithm for Rotationally Invariant $\ell_1$-Norm Principal Component Analysis

TL;DR

This paper introduces a novel linearly convergent algorithm for rotationally invariant -norm PCA, addressing a less-studied problem with theoretical guarantees and empirical validation on synthetic and real data.

Contribution

It proposes a proximal alternating linearized minimization method with nonlinear extrapolation for rotationally invariant -norm PCA, proving linear convergence and critical point optimality.

Findings

The algorithm converges at least linearly to a critical point.

The method effectively handles datasets with outliers.

Numerical experiments validate theoretical results and demonstrate practical efficacy.

Abstract

To do dimensionality reduction on the datasets with outliers, the $\ell_1$-norm principal component analysis (L1-PCA) as a typical robust alternative of the conventional PCA has enjoyed great popularity over the past years. In this work, we consider a rotationally invariant L1-PCA, which is hardly studied in the literature. To tackle it, we propose a proximal alternating linearized minimization method with a nonlinear extrapolation for solving its two-block reformulation. Moreover, we show that the proposed method converges at least linearly to a limiting critical point of the reformulated problem. Such a point is proved to be a critical point of the original problem under a condition imposed on the step size. Finally, we conduct numerical experiments on both synthetic and real datasets to support our theoretical developments and demonstrate the efficacy of our approach.