Rank-Constrained Least-Squares: Prediction and Inference

TL;DR

This paper develops a nearly optimal prediction method for high-dimensional trace regression with low-rank matrices, providing theoretical guarantees and empirical evaluation of a permutation test for low-rank signals without restrictive assumptions.

Contribution

It introduces a new analysis of rank-constrained least-squares in high dimensions, including risk bounds, a power analysis for permutation tests, and practical algorithms for empirical evaluation.

Findings

Nearly optimal in-sample prediction risk bound established.

Permutation test achieves non-trivial power without eigenvalue assumptions.

Empirical results validate the theoretical predictions.

Abstract

In this work, we focus on the high-dimensional trace regression model with a low-rank coefficient matrix. We establish a nearly optimal in-sample prediction risk bound for the rank-constrained least-squares estimator under no assumptions on the design matrix. Lying at the heart of the proof is a covering number bound for the family of projection operators corresponding to the subspaces spanned by the design. By leveraging this complexity result, we perform a power analysis for a permutation test on the existence of a low-rank signal under the high-dimensional trace regression model. We show that the permutation test based on the rank-constrained least-squares estimator achieves non-trivial power with no assumptions on the minimum (restricted) eigenvalue of the covariance matrix of the design. Finally, we use alternating minimization to approximately solve the rank-constrained least-squares problem to evaluate its empirical in-sample prediction risk and power of the resulting permutation test in our numerical study.