Variable selection in multivariate linear models with high-dimensional covariance matrix estimation

TL;DR

This paper introduces a new variable selection method for multivariate linear models that incorporates high-dimensional covariance matrix estimation into a Lasso framework, improving selection accuracy.

Contribution

It proposes a novel approach combining covariance matrix estimation with Lasso for better variable selection in multivariate models, with theoretical guarantees and practical implementation.

Findings

Improved variable selection performance over existing methods.

Theoretical conditions for covariance estimator consistency.

Efficient implementation in R package MultiVarSel.

Abstract

In this paper, we propose a novel variable selection approach in the framework of multivariate linear models taking into account the dependence that may exist between the responses. It consists in estimating beforehand the covariance matrix of the responses and to plug this estimator in a Lasso criterion, in order to obtain a sparse estimator of the coefficient matrix. The properties of our approach are investigated both from a theoretical and a numerical point of view. More precisely, we give general conditions that the estimators of the covariance matrix and its inverse have to satisfy in order to recover the positions of the null and non null entries of the coefficient matrix when the size of the covariance matrix is not fixed and can tend to infinity. We prove that these conditions are satisfied in the particular case of some Toeplitz matrices. Our approach is implemented in the R package MultiVarSel available from the Comprehensive R Archive Network (CRAN) and is very attractive since it benefits from a low computational load. We also assess the performance of our methodology using synthetic data and compare it with alternative approaches. Our numerical experiments show that including the estimation of the covariance matrix in the Lasso criterion dramatically improves the variable selection performance in many cases.