ALPCAH: Subspace Learning for Sample-wise Heteroscedastic Data

TL;DR

ALPCAH is a novel subspace learning method that estimates sample-wise noise variances to improve low-rank data structure recovery in heteroscedastic datasets without prior distributional assumptions.

Contribution

It introduces ALPCAH, a heteroscedastic PCA method that estimates individual noise variances and employs a soft rank constraint, without needing known noise levels or subspace dimension.

Findings

ALPCAH outperforms existing algorithms in heteroscedastic data scenarios.

The matrix factorized version LR-ALPCAH is faster and more memory-efficient.

Experiments on real data validate the effectiveness of accounting for heteroscedasticity.

Abstract

Principal component analysis (PCA) is a key tool in the field of data dimensionality reduction. However, some applications involve heterogeneous data that vary in quality due to noise characteristics associated with each data sample. Heteroscedastic methods aim to deal with such mixed data quality. This paper develops a subspace learning method, named ALPCAH, that can estimate the sample-wise noise variances and use this information to improve the estimate of the subspace basis associated with the low-rank structure of the data. Our method makes no distributional assumptions of the low-rank component and does not assume that the noise variances are known. Further, this method uses a soft rank constraint that does not require subspace dimension to be known. Additionally, this paper develops a matrix factorized version of ALPCAH, named LR-ALPCAH, that is much faster and more memory efficient at the cost of requiring subspace dimension to be known or estimated. Simulations and real data experiments show the effectiveness of accounting for data heteroscedasticity compared to existing algorithms. Code available at https://github.com/javiersc1/ALPCAH.