Bias-Aware Inference in Regularized Regression Models

TL;DR

This paper introduces bias-aware inference methods for regularized regression models that effectively balance bias and variance, providing near-optimal confidence intervals under various error conditions and high-dimensional settings.

Contribution

It develops a class of estimators and confidence intervals that explicitly account for bias, achieving near-optimal finite-sample and asymptotic properties in high-dimensional regression.

Findings

Estimators solve a bias-variance tradeoff effectively.

Confidence intervals are bias-aware and adapt to heteroskedastic errors.

Methods perform well in simulations and empirical data.

Abstract

We consider inference on a scalar regression coefficient under a constraint on the magnitude of the control coefficients. A class of estimators based on a regularized propensity score regression is shown to exactly solve a tradeoff between worst-case bias and variance. We derive confidence intervals (CIs) based on these estimators that are bias-aware: they account for the possible bias of the estimator. Under homoskedastic Gaussian errors, these estimators and CIs are near-optimal in finite samples for MSE and CI length. We also provide conditions for asymptotic validity of the CI with unknown and possibly heteroskedastic error distribution, and derive novel optimal rates of convergence under high-dimensional asymptotics that allow the number of regressors to increase more quickly than the number of observations. Extensive simulations and an empirical application illustrate the performance of our methods.