Selecting Penalty Parameters of High-Dimensional M-Estimators using Bootstrapping after Cross-Validation

TL;DR

This paper introduces a novel bootstrapping after cross-validation method for selecting penalty parameters in high-dimensional L1-penalized M-estimators, improving estimation and inference accuracy.

Contribution

It proposes a new penalty selection technique that enhances high-dimensional M-estimator performance and provides theoretical convergence rates.

Findings

Method outperforms traditional cross-validation in estimation accuracy.

Approach improves inference quality in high-dimensional settings.

Reproduces and confirms findings in a real-world police use of force study.

Abstract

We develop a new method for selecting the penalty parameter for $\ell_{1}$-penalized M-estimators in high dimensions, which we refer to as bootstrapping after cross-validation. We derive rates of convergence for the corresponding $\ell_1$-penalized M-estimator and also for the post-$\ell_1$-penalized M-estimator, which refits the non-zero entries of the former estimator without penalty in the criterion function. We demonstrate via simulations that our methods are not dominated by cross-validation in terms of estimation errors and can outperform cross-validation in terms of inference. As an empirical illustration, we revisit Fryer Jr (2019), who investigated racial differences in police use of force, and confirm his findings.