Estimation of (near) low-rank matrices with noise and high-dimensional scaling

TL;DR

This paper investigates the estimation of low-rank or near low-rank matrices from noisy high-dimensional data using nuclear norm regularization, providing non-asymptotic error bounds and demonstrating their effectiveness through simulations.

Contribution

It offers the first non-asymptotic analysis of nuclear norm regularized estimators in high-dimensional settings with noisy observations, applicable to various matrix models.

Findings

Non-asymptotic Frobenius norm error bounds derived.

Theoretical predictions match simulation results.

Applicable to multiple matrix estimation problems.

Abstract

High-dimensional inference refers to problems of statistical estimation in which the ambient dimension of the data may be comparable to or possibly even larger than the sample size. We study an instance of high-dimensional inference in which the goal is to estimate a matrix $\Theta^* \in \real^{k \times p}$ on the basis of $N$ noisy observations, and the unknown matrix $\Theta^*$ is assumed to be either exactly low rank, or ``near'' low-rank, meaning that it can be well-approximated by a matrix with low rank. We consider an $M$-estimator based on regularization by the trace or nuclear norm over matrices, and analyze its performance under high-dimensional scaling. We provide non-asymptotic bounds on the Frobenius norm error that hold for a general class of noisy observation models, and then illustrate their consequences for a number of specific matrix models, including low-rank multivariate or multi-task regression, system identification in vector autoregressive processes, and recovery of low-rank matrices from random projections. Simulation results show excellent agreement with the high-dimensional scaling of the error predicted by our theory.