Rank penalized estimators for high-dimensional matrices

TL;DR

This paper introduces a new rank penalized estimator for high-dimensional matrices within the trace regression model, providing theoretical guarantees and applying it to matrix completion and regression tasks.

Contribution

It proposes a novel rank penalized estimator with oracle inequalities and rank bounds, simplifying to singular value hard thresholding in key applications.

Findings

Estimator achieves oracle inequalities for prediction error

Provides upper bounds for the estimator's rank

Simplifies to singular value hard thresholding in matrix completion and regression

Abstract

In this paper we consider the trace regression model. Assume that we observe a small set of entries or linear combinations of entries of an unknown matrix $A_0$ corrupted by noise. We propose a new rank penalized estimator of $A_0$. For this estimator we establish general oracle inequality for the prediction error both in probability and in expectation. We also prove upper bounds for the rank of our estimator. Then, we apply our general results to the problems of matrix completion and matrix regression. In these cases our estimator has a particularly simple form: it is obtained by hard thresholding of the singular values of a matrix constructed from the observations.