Generative Quantile Regression with Variability Penalty

TL;DR

This paper introduces PGQR, a deep generative model for joint quantile estimation that captures complex distributional features and avoids common issues like crossing quantiles and variability vanishing, using a novel penalty and neural network design.

Contribution

We propose PGQR, a novel deep generative model with a variability penalty and partial monotonic neural networks for efficient joint quantile estimation.

Findings

Effective in capturing multimodality and skewness.

Avoids quantile crossing with PMNN.

Single optimization for multiple quantiles.

Abstract

Quantile regression and conditional density estimation can reveal structure that is missed by mean regression, such as multimodality and skewness. In this paper, we introduce a deep learning generative model for joint quantile estimation called Penalized Generative Quantile Regression (PGQR). Our approach simultaneously generates samples from many random quantile levels, allowing us to infer the conditional distribution of a response variable given a set of covariates. Our method employs a novel variability penalty to avoid the problem of vanishing variability, or memorization, in deep generative models. Further, we introduce a new family of partial monotonic neural networks (PMNN) to circumvent the problem of crossing quantile curves. A major benefit of PGQR is that it can be fit using a single optimization, thus bypassing the need to repeatedly train the model at multiple quantile levels or use computationally expensive cross-validation to tune the penalty parameter. We illustrate the efficacy of PGQR through extensive simulation studies and analysis of real datasets. Code to implement our method is available at https://github.com/shijiew97/PGQR.