A comparison of some conformal quantile regression methods

TL;DR

This paper compares two conformal quantile regression methods, proving their asymptotic efficiency and empirically evaluating their performance on simulated and real datasets, highlighting differences in prediction interval tightness.

Contribution

It provides a theoretical comparison of two recent conformal quantile regression methods and evaluates their finite-sample performance.

Findings

Romano et al. (2019) method generally produces tighter intervals

Both methods are asymptotically efficient under certain conditions

Performance varies with sample size and tuning parameters

Abstract

We compare two recently proposed methods that combine ideas from conformal inference and quantile regression to produce locally adaptive and marginally valid prediction intervals under sample exchangeability (Romano et al., 2019; Kivaranovic et al., 2019). First, we prove that these two approaches are asymptotically efficient in large samples, under some additional assumptions. Then we compare them empirically on simulated and real data. Our results demonstrate that the method in Romano et al. (2019) typically yields tighter prediction intervals in finite samples. Finally, we discuss how to tune these procedures by fixing the relative proportions of observations used for training and conformalization.