High-dimensional inference via hybrid orthogonalization

TL;DR

This paper introduces a hybrid orthogonalization method for high-dimensional inference that relaxes sparsity constraints by residualizing the design matrix, enabling more practical and robust statistical testing.

Contribution

It proposes the HOT technique, which combines strict and relaxed orthogonalization to improve inference in high-dimensional sparse models with identifiable features.

Findings

HOT achieves asymptotic normality under less restrictive conditions.

Simulation results demonstrate HOT's superior performance.

Application to stock data confirms practical utility.

Abstract

The past decade has witnessed a surge of endeavors in statistical inference for high-dimensional sparse regression, particularly via de-biasing or relaxed orthogonalization. Nevertheless, these techniques typically require a more stringent sparsity condition than needed for estimation consistency, which seriously limits their practical applicability. To alleviate such constraint, we propose to exploit the identifiable features to residualize the design matrix before performing debiasing-based inference over the parameters of interest. This leads to a hybrid orthogonalization (HOT) technique that performs strict orthogonalization against the identifiable features but relaxed orthogonalization against the others. Under an approximately sparse model with a mixture of identifiable and unidentifiable signals, we establish the asymptotic normality of the HOT test statistic while accommodating as many identifiable signals as consistent estimation allows. The efficacy of the proposed test is also demonstrated through simulation and analysis of a stock market dataset.