Robust Principal Component Analysis via Discriminant Sample Weight Learning

TL;DR

This paper introduces a robust PCA method that assigns discriminant weights to samples, iteratively learning the mean and projection matrix to effectively mitigate outliers' influence.

Contribution

It proposes a novel iterative algorithm that learns sample weights, mean, and projection matrix simultaneously, enhancing robustness against outliers in PCA.

Findings

Effective outlier mitigation demonstrated on multiple datasets

Improved accuracy in estimating PCA components with outliers

Hierarchical weight learning enhances robustness

Abstract

Principal component analysis (PCA) is a classical feature extraction method, but it may be adversely affected by outliers, resulting in inaccurate learning of the projection matrix. This paper proposes a robust method to estimate both the data mean and the PCA projection matrix by learning discriminant sample weights from data containing outliers. Each sample in the dataset is assigned a weight, and the proposed algorithm iteratively learns the weights, the mean, and the projection matrix, respectively. Specifically, when the mean and the projection matrix are available, via fine-grained analysis of outliers, a weight for each sample is learned hierarchically so that outliers have small weights while normal samples have large weights. With the learned weights available, a weighted optimization problem is solved to estimate both the data mean and the projection matrix. Because the learned weights discriminate outliers from normal samples, the adverse influence of outliers is mitigated due to the corresponding small weights. Experiments on toy data, UCI dataset, and face dataset demonstrate the effectiveness of the proposed method in estimating the mean and the projection matrix from the data containing outliers.