fastkqr: A Fast Algorithm for Kernel Quantile Regression

TL;DR

fastkqr is a novel, efficient algorithm for kernel quantile regression that produces exact quantiles quickly, making robust, heterogeneous learning more computationally feasible.

Contribution

The paper introduces fastkqr, a finite smoothing algorithm with spectral acceleration for kernel quantile regression, including a data-driven crossing penalty extension.

Findings

Fastkqr achieves up to ten times faster computation than existing methods.

It maintains accuracy comparable to state-of-the-art algorithms.

The method is implemented in an open-source R package.

Abstract

Quantile regression is a powerful tool for robust and heterogeneous learning that has seen applications in a diverse range of applied areas. However, its broader application is often hindered by the substantial computational demands arising from the non-smooth quantile loss function. In this paper, we introduce a novel algorithm named fastkqr, which significantly advances the computation of quantile regression in reproducing kernel Hilbert spaces. The core of fastkqr is a finite smoothing algorithm that magically produces exact regression quantiles, rather than approximations. To further accelerate the algorithm, we equip fastkqr with a novel spectral technique that carefully reutilizes matrix computations. In addition, we extend fastkqr to accommodate a flexible kernel quantile regression with a data-driven crossing penalty, addressing the interpretability challenges of crossing quantile curves at multiple levels. We have implemented fastkqr in a publicly available R package. Extensive simulations and real applications show that fastkqr matches the accuracy of state-of-the-art algorithms but can operate up to an order of magnitude faster.