Nonparametric Quantile Regression: Non-Crossing Constraints and Conformal Prediction

TL;DR

This paper introduces a deep neural network-based nonparametric quantile regression method with a novel non-crossing penalty, providing optimal error bounds and adaptive conformal prediction intervals with strong theoretical guarantees and practical effectiveness.

Contribution

It develops a computationally feasible non-crossing penalty for deep neural network quantile regression with optimal convergence rates and constructs adaptive conformal prediction intervals.

Findings

Error bounds achieve minimax optimal rate.

Prediction intervals are valid and accurate.

Method performs well in simulations and real data.

Abstract

We propose a nonparametric quantile regression method using deep neural networks with a rectified linear unit penalty function to avoid quantile crossing. This penalty function is computationally feasible for enforcing non-crossing constraints in multi-dimensional nonparametric quantile regression. We establish non-asymptotic upper bounds for the excess risk of the proposed nonparametric quantile regression function estimators. Our error bounds achieve optimal minimax rate of convergence for the Holder class, and the prefactors of the error bounds depend polynomially on the dimension of the predictor, instead of exponentially. Based on the proposed non-crossing penalized deep quantile regression, we construct conformal prediction intervals that are fully adaptive to heterogeneity. The proposed prediction interval is shown to have good properties in terms of validity and accuracy under reasonable conditions. We also derive non-asymptotic upper bounds for the difference of the lengths between the proposed non-crossing conformal prediction interval and the theoretically oracle prediction interval. Numerical experiments including simulation studies and a real data example are conducted to demonstrate the effectiveness of the proposed method.