Accurate Inference for Penalized Logistic Regression

TL;DR

This paper introduces a two-step method for high-dimensional logistic regression that improves inference accuracy by reducing bias, outperforming existing approaches in finite sample scenarios.

Contribution

A novel two-step procedure combining Lasso-based variable selection and bias correction for more accurate inference in high-dimensional logistic regression.

Findings

Significantly smaller biases than alternative methods in finite samples

Improved inference performance demonstrated through numerical studies

Effective application to alcohol consumption data analysis

Abstract

Inference for high-dimensional logistic regression models using penalized methods has been a challenging research problem. As an illustration, a major difficulty is the significant bias of the Lasso estimator, which limits its direct application in inference. Although various bias corrected Lasso estimators have been proposed, they often still exhibit substantial biases in finite samples, undermining their inference performance. These finite sample biases become particularly problematic in one-sided inference problems, such as one-sided hypothesis testing. This paper proposes a novel two-step procedure for accurate inference in high-dimensional logistic regression models. In the first step, we propose a Lasso-based variable selection method to select a suitable submodel of moderate size for subsequent inference. In the second step, we introduce a bias corrected estimator to fit the selected submodel. We demonstrate that the resulting estimator from this two-step procedure has a small bias order and enables accurate inference. Numerical studies and an analysis of alcohol consumption data are included, where our proposed method is compared to alternative approaches. Our results indicate that the proposed method exhibits significantly smaller biases than alternative methods in finite samples, thereby leading to improved inference performance.