Recovering PCA from Hybrid-$(\ell_1,\ell_2)$ Sparse Sampling of Data Elements

TL;DR

This paper introduces a hybrid sampling algorithm combining $\, ext{ extlbrackdbl}\, ext{ extlbrackdbl} ext{ and }\ell_2$ methods to efficiently recover a data matrix's principal components from limited element-wise samples, outperforming existing methods.

Contribution

The paper proposes a novel hybrid sampling algorithm that leverages both $\, ext{ extlbrackdbl}\, ext{ extlbrackdbl} ext{ and }\ell_2$ sampling for matrix recovery, providing theoretical guarantees and practical efficiency.

Findings

The hybrid algorithm achieves near-PCA reconstruction with sublinear samples.

It outperforms pure $\, ext{ extlbrackdbl} ext{ or }\ell_2$ sampling in accuracy.

Experimental results validate the theoretical advantages.

Abstract

This paper addresses how well we can recover a data matrix when only given a few of its elements. We present a randomized algorithm that element-wise sparsifies the data, retaining only a few its elements. Our new algorithm independently samples the data using sampling probabilities that depend on both the squares ($\ell_2$ sampling) and absolute values ($\ell_1$ sampling) of the entries. We prove that the hybrid algorithm recovers a near-PCA reconstruction of the data from a sublinear sample-size: hybrid-($\ell_1,\ell_2$) inherits the $\ell_2$-ability to sample the important elements as well as the regularization properties of $\ell_1$ sampling, and gives strictly better performance than either $\ell_1$ or $\ell_2$ on their own. We also give a one-pass version of our algorithm and show experiments to corroborate the theory.