Advanced Algorithms for Penalized Quantile and Composite Quantile Regression

TL;DR

This paper introduces new algorithms for high-dimensional quantile and composite quantile regression, addressing the lack of scalable methods, and demonstrates their effectiveness through simulations and implementation in an R package.

Contribution

It develops and compares ADMM, MM, and CD algorithms for penalized quantile regression, filling a gap in high-dimensional, regularized quantile regression methods.

Findings

MM and CD algorithms outperform existing methods in different data settings.

The proposed methods are implemented in the cqrReg R package.

ADMM algorithm shows promise for parallelization and scalability.

Abstract

In this paper, we discuss a family of robust, high-dimensional regression models for quantile and composite quantile regression, both with and without an adaptive lasso penalty for variable selection. We reformulate these quantile regression problems and obtain estimators by applying the alternating direction method of multipliers (ADMM), majorize-minimization (MM), and coordinate descent (CD) algorithms. Our new approaches address the lack of publicly available methods for (composite) quantile regression, especially for high-dimensional data, both with and without regularization. Through simulation studies, we demonstrate the need for different algorithms applicable to a variety of data settings, which we implement in the cqrReg package for R. For comparison, we also introduce the widely used interior point (IP) formulation and test our methods against the IP algorithms in the existing quantreg package. Our simulation studies show that each of our methods, particularly MM and CD, excel in different settings such as with large or high-dimensional data sets, respectively, and outperform the methods currently implemented in quantreg. The ADMM approach offers specific promise for future developments in its amenability to parallelization and scalability.