A Pathwise Coordinate Descent Algorithm for LASSO Penalized Quantile Regression

TL;DR

This paper introduces a fast, exact, pathwise coordinate descent algorithm for high-dimensional $ ext{L}_1$ penalized quantile regression, overcoming computational challenges of nonsmooth loss functions and improving efficiency over existing methods.

Contribution

The authors develop a novel, exact coordinatewise minimization approach for nonsmooth quantile loss, enabling scalable, fast pathwise $ ext{L}_1$ penalized quantile regression in high dimensions.

Findings

Algorithm outperforms existing methods in speed

Retains estimation accuracy comparable to alternatives

Employs a perturbation strategy to avoid pathwise stagnation

Abstract

$\ell_1$ penalized quantile regression is used in many fields as an alternative to penalized least squares regressions for high-dimensional data analysis. Existing algorithms for penalized quantile regression either use linear programming, which does not scale well in high dimension, or an approximate coordinate descent (CD) which does not solve for exact coordinatewise minimum of the nonsmooth loss function. Further, neither approaches build fast, pathwise algorithms commonly used in high-dimensional statistics to leverage sparsity structure of the problem in large-scale data sets. To avoid the computational challenges associated with the nonsmooth quantile loss, some recent works have even advocated using smooth approximations to the exact problem. In this work, we develop a fast, pathwise coordinate descent algorithm to compute exact $\ell_1$ penalized quantile regression estimates for high-dimensional data. We derive an easy-to-compute exact solution for the coordinatewise nonsmooth loss minimization, which, to the best of our knowledge, has not been reported in the literature. We also employ a random perturbation strategy to help the algorithm avoid getting stuck along the regularization path. In simulated data sets, we show that our algorithm runs substantially faster than existing alternatives based on approximate CD and linear program, while retaining the same level of estimation accuracy.