Efficient Strongly Polynomial Algorithms for Quantile Regression

TL;DR

This paper introduces new strongly polynomial algorithms for quantile regression, significantly improving computational efficiency and providing deterministic and randomized solutions for various dimensions.

Contribution

The paper develops the first strongly polynomial algorithms for quantile regression, including deterministic and randomized methods with improved time complexities for different dimensions.

Findings

Deterministic algorithm with $ ilde{O}(n^{4/3})$ complexity for 2D QR.

Randomized divide-and-conquer algorithm with $O(n \, \log^2 n)$ complexity for 2D QR.

Generalized randomized algorithm with $O(n^{d-1} \, \log^2 n)$ complexity for higher dimensions.

Abstract

Linear Regression is a seminal technique in statistics and machine learning, where the objective is to build linear predictive models between a response (i.e., dependent) variable and one or more predictor (i.e., independent) variables. In this paper, we revisit the classical technique of Quantile Regression (QR), which is statistically a more robust alternative to the other classical technique of Ordinary Least Square Regression (OLS). However, while there exist efficient algorithms for OLS, almost all of the known results for QR are only weakly polynomial. Towards filling this gap, this paper proposes several efficient strongly polynomial algorithms for QR for various settings. For two dimensional QR, making a connection to the geometric concept of $k$-set, we propose an algorithm with a deterministic worst-case time complexity of $\mathcal{O}(n^{4/3} polylog(n))$ and an expected time complexity of $\mathcal{O}(n^{4/3})$ for the randomized version. We also propose a randomized divide-and-conquer algorithm -- RandomizedQR with an expected time complexity of $\mathcal{O}(n\log^2{(n)})$ for two dimensional QR problem. For the general case with more than two dimensions, our RandomizedQR algorithm has an expected time complexity of $\mathcal{O}(n^{d-1}\log^2{(n)})$.