Bootstrap-Based Regularization for Low-Rank Matrix Estimation

TL;DR

This paper introduces a bootstrap-based framework for low-rank matrix estimation that transforms noise models into regularization schemes, producing stable autoencoders that adapt to different noise types and automatically determine rank.

Contribution

The authors propose a novel bootstrap-based regularization method for low-rank matrix estimation that generalizes singular value shrinkage and automatically infers the rank without tuning.

Findings

Equivalent to classical shrinkage under isotropic noise

Produces new estimators for non-isotropic noise like Poisson

Automatically determines low-rank estimates through iteration

Abstract

We develop a flexible framework for low-rank matrix estimation that allows us to transform noise models into regularization schemes via a simple bootstrap algorithm. Effectively, our procedure seeks an autoencoding basis for the observed matrix that is stable with respect to the specified noise model; we call the resulting procedure a stable autoencoder. In the simplest case, with an isotropic noise model, our method is equivalent to a classical singular value shrinkage estimator. For non-isotropic noise models, e.g., Poisson noise, the method does not reduce to singular value shrinkage, and instead yields new estimators that perform well in experiments. Moreover, by iterating our stable autoencoding scheme, we can automatically generate low-rank estimates without specifying the target rank as a tuning parameter.