Principled Inference in Dense High-Dimensional Linear Models via Local Conditional Sparsity

TL;DR

This paper introduces NLNR, a new high-dimensional inference method that localizes inference to a sparse neighborhood, enabling valid inference even with dense coefficients.

Contribution

The paper develops NLNR, a novel framework for coordinatewise inference in dense high-dimensional linear models using local conditional sparsity, with theoretical guarantees and practical algorithms.

Findings

NLNR achieves consistent and asymptotically normal estimators.

The thresholding and boosting variants improve finite-sample performance.

Empirical results demonstrate the method's effectiveness on real data.

Abstract

High-dimensional inference methods often rely on coefficient sparsity, an assumption that can be restrictive when signals are dense but individually weak. In such settings, valid inference may still be possible if the covariates exhibit sparse conditional dependence. Motivated by this observation, we propose Neighborhood-Localized Nested Regression (NLNR), a framework for coordinatewise inference in high-dimensional linear models with potentially dense coefficients. The central idea is to localize inference for a target coefficient to a low-dimensional working regression determined by a Sparse Conditional Neighborhood (SCN) of the target covariate. Specifically, for a given covariate, we estimate its SCN through nodewise $\ell_1$-penalized regression and then fit a regression using only the target covariate and its estimated neighborhood. Under suitable regularity conditions, we establish consistency and asymptotic normality of the resulting estimator. Building on this inferential reduction principle, we further develop a thresholding-based screening procedure with theoretical guarantees and a boosting variant that augments the working model with additional response-relevant covariates to improve finite-sample performance. Extensive simulations and an application to the CCLE dataset demonstrate favorable empirical performance.