Uniform Post Selection Inference for LAD Regression and Other Z-estimation problems

TL;DR

This paper introduces a method for constructing uniformly valid confidence regions for regression coefficients in high-dimensional median regression models, utilizing Neyman's orthogonalization to handle nuisance parameters and extending to general Z-estimation problems.

Contribution

It develops a novel orthogonalization-based approach for uniform inference in high-dimensional median regression and general Z-estimation, achieving semi-parametric efficiency and valid confidence bands.

Findings

Asymptotic normality of the estimator is established uniformly over models.

Constructs simultaneous confidence bands for multiple parameters.

Method extends to high-dimensional Z-estimation with many parameters.

Abstract

We develop uniformly valid confidence regions for regression coefficients in a high-dimensional sparse median regression model with homoscedastic errors. Our methods are based on a moment equation that is immunized against non-regular estimation of the nuisance part of the median regression function by using Neyman's orthogonalization. We establish that the resulting instrumental median regression estimator of a target regression coefficient is asymptotically normally distributed uniformly with respect to the underlying sparse model and is semi-parametrically efficient. We also generalize our method to a general non-smooth Z-estimation framework with the number of target parameters $p_1$ being possibly much larger than the sample size $n$. We extend Huber's results on asymptotic normality to this setting, demonstrating uniform asymptotic normality of the proposed estimators over $p_1$-dimensional rectangles, constructing simultaneous confidence bands on all of the $p_1$ target parameters, and establishing asymptotic validity of the bands uniformly over underlying approximately sparse models. Keywords: Instrument; Post-selection inference; Sparsity; Neyman's Orthogonal Score test; Uniformly valid inference; Z-estimation.