Tuning parameter calibration for $\ell_1$-regularized logistic regression

TL;DR

This paper introduces a new calibration scheme for $\, ext{l}_1$-penalized logistic regression that provides finite sample guarantees and improves feature selection accuracy in high-dimensional classification tasks.

Contribution

The paper proposes a novel, test-based calibration method for $\, ext{l}_1$-penalized logistic regression with optimal guarantees, enhancing feature selection in high-dimensional data.

Findings

Outperforms existing calibration methods in simulations.

Provides finite sample guarantees for feature selection.

Efficient implementation and competitive real data performance.

Abstract

Feature selection is a standard approach to understanding and modeling high-dimensional classification data, but the corresponding statistical methods hinge on tuning parameters that are difficult to calibrate. In particular, existing calibration schemes in the logistic regression framework lack any finite sample guarantees. In this paper, we introduce a novel calibration scheme for $\ell_1$-penalized logistic regression. It is based on simple tests along the tuning parameter path and is equipped with optimal guarantees for feature selection. It is also amenable to easy and efficient implementations, and it rivals or outmatches existing methods in simulations and real data applications.