Self-paced Principal Component Analysis

TL;DR

This paper introduces Self-paced PCA, a novel method that adaptively learns from simple to complex samples to improve robustness against noise and outliers in dimensionality reduction tasks.

Contribution

It proposes a self-paced learning framework for PCA that dynamically assesses sample complexity and iteratively filters out outliers, enhancing robustness and performance.

Findings

Significantly improves state-of-the-art results on benchmark datasets.

Effectively filters out noise and outliers during learning.

Demonstrates theoretical soundness and practical effectiveness.

Abstract

Principal Component Analysis (PCA) has been widely used for dimensionality reduction and feature extraction. Robust PCA (RPCA), under different robust distance metrics, such as l1-norm and l2, p-norm, can deal with noise or outliers to some extent. However, real-world data may display structures that can not be fully captured by these simple functions. In addition, existing methods treat complex and simple samples equally. By contrast, a learning pattern typically adopted by human beings is to learn from simple to complex and less to more. Based on this principle, we propose a novel method called Self-paced PCA (SPCA) to further reduce the effect of noise and outliers. Notably, the complexity of each sample is calculated at the beginning of each iteration in order to integrate samples from simple to more complex into training. Based on an alternating optimization, SPCA finds an optimal projection matrix and filters out outliers iteratively. Theoretical analysis is presented to show the rationality of SPCA. Extensive experiments on popular data sets demonstrate that the proposed method can improve the state of-the-art results considerably.