A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models

TL;DR

This paper introduces a general framework for hypothesis testing and confidence regions in high dimensional models, using decorrelated score functions to handle nuisance parameters, applicable across various models and settings.

Contribution

It develops a universal approach for high dimensional inference that is applicable to a wide range of models, including under misspecification and with generalized loss functions.

Findings

The decorrelated score test controls type I error and has good power.

The method achieves semiparametric efficiency for estimators.

Extensions handle high dimensional null hypotheses and model misspecification.

Abstract

We consider the problem of uncertainty assessment for low dimensional components in high dimensional models. Specifically, we propose a decorrelated score function to handle the impact of high dimensional nuisance parameters. We consider both hypothesis tests and confidence regions for generic penalized M-estimators. Unlike most existing inferential methods which are tailored for individual models, our approach provides a general framework for high dimensional inference and is applicable to a wide range of applications. From the testing perspective, we develop general theorems to characterize the limiting distributions of the decorrelated score test statistic under both null hypothesis and local alternatives. These results provide asymptotic guarantees on the type I errors and local powers of the proposed test. Furthermore, we show that the decorrelated score function can be used to construct point and confidence region estimators that are semiparametrically efficient. We also generalize this framework to broaden its applications. First, we extend it to handle high dimensional null hypothesis, where the number of parameters of interest can increase exponentially fast with the sample size. Second, we establish the theory for model misspecification. Third, we go beyond the likelihood framework, by introducing the generalized score test based on general loss functions. Thorough numerical studies are conducted to back up the developed theoretical results.