Sparse Quantile Regression

TL;DR

This paper introduces $ ext{L}_0$-penalized and constrained quantile regression methods, providing theoretical guarantees, efficient algorithms, and demonstrating their superior sparsity and comparable accuracy in simulations and real data applications.

Contribution

It develops nearly minimax-optimal $ ext{L}_0$-based quantile regression estimators with theoretical bounds, scalable algorithms, and empirical validation.

Findings

Nearly minimax-optimal convergence rates.

Sparse estimators outperform $ ext{L}_1$-penalized methods.

Effective in high-dimensional real data applications.

Abstract

We consider both $\ell _{0}$-penalized and $\ell _{0}$-constrained quantile regression estimators. For the $\ell _{0}$-penalized estimator, we derive an exponential inequality on the tail probability of excess quantile prediction risk and apply it to obtain non-asymptotic upper bounds on the mean-square parameter and regression function estimation errors. We also derive analogous results for the $\ell _{0}$-constrained estimator. The resulting rates of convergence are nearly minimax-optimal and the same as those for $\ell _{1}$-penalized and non-convex penalized estimators. Further, we characterize expected Hamming loss for the $\ell _{0}$-penalized estimator. We implement the proposed procedure via mixed integer linear programming and also a more scalable first-order approximation algorithm. We illustrate the finite-sample performance of our approach in Monte Carlo experiments and its usefulness in a real data application concerning conformal prediction of infant birth weights (with $n\approx 10^{3}$ and up to $p>10^{3}$). In sum, our $\ell _{0}$-based method produces a much sparser estimator than the $\ell _{1}$-penalized and non-convex penalized approaches without compromising precision.