Joint variable and rank selection for parsimonious estimation of high-dimensional matrices

TL;DR

This paper introduces a joint variable and rank selection method for high-dimensional multivariate regression, improving prediction accuracy by simultaneously reducing dimensions through novel penalized estimators.

Contribution

It proposes a new class of sparse multivariate regression models with joint row and rank restrictions, along with estimators that adapt to unknown sparsity and achieve fast convergence.

Findings

Estimators adapt to unknown sparsity levels.

Significant prediction accuracy improvements shown in simulations.

Effective application demonstrated on real data.

Abstract

We propose dimension reduction methods for sparse, high-dimensional multivariate response regression models. Both the number of responses and that of the predictors may exceed the sample size. Sometimes viewed as complementary, predictor selection and rank reduction are the most popular strategies for obtaining lower-dimensional approximations of the parameter matrix in such models. We show in this article that important gains in prediction accuracy can be obtained by considering them jointly. We motivate a new class of sparse multivariate regression models, in which the coefficient matrix has low rank and zero rows or can be well approximated by such a matrix. Next, we introduce estimators that are based on penalized least squares, with novel penalties that impose simultaneous row and rank restrictions on the coefficient matrix. We prove that these estimators indeed adapt to the unknown matrix sparsity and have fast rates of convergence. We support our theoretical results with an extensive simulation study and two data analyses.