On Unbiased Low-Rank Approximation with Minimum Distortion

TL;DR

This paper introduces an algorithm for unbiased low-rank matrix approximation that minimizes expected Frobenius norm error, extending sparsification techniques to matrices and proving optimality against known bounds.

Contribution

It presents a novel unbiased low-rank approximation algorithm for matrices, inspired by vector sparsification methods, with proven optimality in error minimization.

Findings

Algorithm achieves unbiased low-rank approximation with minimal distortion.

Matches the theoretical lower bound for approximation error.

Provides a practical method for matrix approximation with optimal error bounds.

Abstract

We describe an algorithm for sampling a low-rank random matrix $Q$ that best approximates a fixed target matrix $P\in\mathbb{C}^{n\times m}$ in the following sense: $Q$ is unbiased, i.e., $\mathbb{E}[Q] = P$; $\mathsf{rank}(Q)\leq r$; and $Q$ minimizes the expected Frobenius norm error $\mathbb{E}\|P-Q\|_F^2$. Our algorithm mirrors the solution to the efficient unbiased sparsification problem for vectors, except applied to the singular components of the matrix $P$. Optimality is proven by showing that our algorithm matches the error from an existing lower bound.