Stability of a Generalized Debiased Lasso with Applications to Resampling-Based Variable Selection

TL;DR

This paper introduces a generalized debiased Lasso estimator based on a stability principle, enabling efficient resampling-based variable selection with theoretical accuracy guarantees under certain conditions.

Contribution

It develops a simple update formula for the estimator under perturbations, reducing computational costs in variable selection methods.

Findings

Estimator admits a simple update formula for perturbed columns

Approximation is asymptotically accurate for most coordinates

Significantly reduces computational cost of resampling procedures

Abstract

We propose a generalized debiased Lasso estimator based on a stability principle. When a single column of the design matrix is perturbed, the estimator admits a simple update formula that can be computed from the original solution. Under sub-Gaussian designs with well-conditioned covariance, this approximation is asymptotically accurate for all but a vanishing fraction of coordinates in the proportional growth regime. The proof relies on concentration and anti-concentration arguments to control error terms and sign changes. In contrast, establishing comparable distributional limits (e.g., Gaussianity) under similar assumptions remains open. As an application, we show that the approximation significantly reduces the computational cost of resampling-based variable selection procedures, including the conditional randomization test and a local knockoff filter.