Indirect multivariate response linear regression

TL;DR

This paper introduces a novel class of estimators for multivariate response linear regression that leverage joint normality assumptions, focusing on inverse regression parameters to improve estimation accuracy.

Contribution

It proposes a new estimation framework based on inverse regression, allowing for sparsity and rank deficiency assumptions without requiring sparsity in the forward regression coefficients.

Findings

Estimators outperform competitors in simulation studies.

Convergence rate bounds are established for the proposed estimators.

Two examples demonstrate practical advantages of the methods.

Abstract

We propose a new class of estimators of the multivariate response linear regression coefficient matrix that exploits the assumption that the response and predictors have a joint multivariate Normal distribution. This allows us to indirectly estimate the regression coefficient matrix through shrinkage estimation of the parameters of the inverse regression, or the conditional distribution of the predictors given the responses. We establish a convergence rate bound for estimators in our class and we study two examples. The first example estimator exploits an assumption that the inverse regression's coefficient matrix is sparse. The second example estimator exploits an assumption that the inverse regression's coefficient matrix is rank deficient. These estimators do not require the popular assumption that the forward regression coefficient matrix is sparse or has small Frobenius norm. Using simulation studies, we show that our example estimators outperform relevant competitors for some data generating models.