Likelihood Ratio Test in Multivariate Linear Regression: from Low to High Dimension

TL;DR

This paper investigates the limitations and power of the likelihood ratio test in high-dimensional multivariate linear regression, proposing corrected distributions, a power-enhanced test, and a dimension reduction approach for cases where predictors exceed sample size.

Contribution

It provides the first asymptotic boundary for LRT failure in high dimensions, develops corrected distributions, and introduces a two-step procedure for $p>n$ scenarios.

Findings

Classical LRT fails when response and predictor dimensions grow with sample size.

Corrected limiting distribution of LRT is derived for general asymptotic regimes.

Proposed two-step testing procedure performs well in high-dimensional settings.

Abstract

Multivariate linear regressions are widely used statistical tools in many applications to model the associations between multiple related responses and a set of predictors. To infer such associations, it is often of interest to test the structure of the regression coefficients matrix, and the likelihood ratio test (LRT) is one of the most popular approaches in practice. Despite its popularity, it is known that the classical $\chi^2$ approximations for LRTs often fail in high-dimensional settings, where the dimensions of responses and predictors $(m,p)$ are allowed to grow with the sample size $n$. Though various corrected LRTs and other test statistics have been proposed in the literature, the fundamental question of when the classic LRT starts to fail is less studied, an answer to which would provide insights for practitioners, especially when analyzing data with $m/n$ and $p/n$ small but not negligible. Moreover, the power performance of the LRT in high-dimensional data analysis remains underexplored. To address these issues, the first part of this work gives the asymptotic boundary where the classical LRT fails and develops the corrected limiting distribution of the LRT for a general asymptotic regime. The second part of this work further studies the test power of the LRT in the high-dimensional setting. The result not only advances the current understanding of asymptotic behavior of the LRT under alternative hypothesis, but also motivates the development of a power-enhanced LRT. The third part of this work considers the setting with $p>n$, where the LRT is not well-defined. We propose a two-step testing procedure by first performing dimension reduction and then applying the proposed LRT. Theoretical properties are developed to ensure the validity of the proposed method. Numerical studies are also presented to demonstrate its good performance.