Generalized High-Dimensional Trace Regression via Nuclear Norm Regularization

TL;DR

This paper introduces a generalized high-dimensional trace regression model with nuclear norm regularization, extending traditional methods to accommodate various response types and matrix regressors, with theoretical guarantees and practical benefits.

Contribution

It develops a unified theoretical framework for generalized trace regression with nuclear norm regularization, deriving minimax optimal rates for diverse applications.

Findings

Estimator achieves minimax rates up to logarithmic factors.

Numerical studies confirm theoretical rates and advantages over linear trace regression.

Nuclear norm regularization benefits stock return prediction and image classification.

Abstract

We study the generalized trace regression with a near low-rank regression coefficient matrix, which extends notion of sparsity for regression coefficient vectors. Specifically, given a matrix covariate $X$, the probability density function $f(Y\vert X)=c(Y)\exp{(\phi^{-1}\left[-Y\eta^* + b(\eta^*)\right])}$, where $\eta^*=tr({\Theta^*}^TX)$. This model accommodates various types of responses and embraces many important problem setups such as reduced-rank regression, matrix regression that accommodates a panel of regressors, matrix completion, among others. We estimate $\Theta^*$ through minimizing empirical negative log-likelihood plus nuclear norm penalty. We first establish a general theory and then for each specific problem, we derive explicitly the statistical rate of the proposed estimator. They all match the minimax rates in the linear trace regression up to logarithmic factors. Numerical studies confirm the rates we established and demonstrate the advantage of generalized trace regression over linear trace regression when the response is dichotomous. We also show the benefit of incorporating nuclear norm regularization in dynamic stock return prediction and in image classification.