A PyTorch Framework for Scalable Non-Crossing Quantile Regression

TL;DR

This paper introduces a scalable PyTorch-based framework for non-crossing quantile regression that ensures monotonicity, reduces computational complexity, and extends to neural networks, with practical applications demonstrated in education and other fields.

Contribution

It presents CJQR-ALM, a novel scalable method combining augmented Lagrangian and differentiable loss for non-crossing quantile regression, suitable for large datasets and neural network models.

Findings

Achieves near-zero crossing rates on large datasets within minutes.

Reduces computational complexity from cubic to linear in data size.

Maintains high accuracy with minimal RMSE increase compared to unconstrained methods.

Abstract

Quantile regression is fundamental to distributional modeling, yet independent estimation of multiple quantiles frequently produces crossing -- where estimated quantile functions violate monotonicity, implying impossible negative probability densities. While Constrained Joint Quantile Regression (CJQR) elegantly enforces non-crossing by construction, existing formulations via Linear Programming exhibit $O((qn)^3)$ complexity, rendering them impractical for large-scale applications. We present the first scalable solution using PyTorch automatic differentiation: \textbf{CJQR-ALM}, combining the \textbf{Augmented Lagrangian Method} with \textbf{differentiable pinball loss} and \textbf{L-BFGS} optimization. Our approach reduces computational complexity to $O(n)$, achieving near-zero crossing rates on datasets exceeding 70,000 observations within minutes. The differentiable formulation naturally extends to neural network architectures for non-linear conditional quantile estimation. Application to Student Growth Percentile calculations demonstrates practical utility for educational assessment, while simulation studies show negligible accuracy cost (RMSE increase $\approx 2.4$ points) relative to unconstrained estimation -- a favorable trade-off for applications requiring valid probability statements across finance, healthcare, and engineering.