coverforest: Conformal Predictions with Random Forest in Python

TL;DR

coverforest is a Python package that efficiently implements conformal prediction methods for random forests, providing reliable uncertainty quantification with faster computation times than existing tools.

Contribution

it introduces coverforest, a Python library that optimizes conformal prediction for random forests using parallel and Cython techniques, enabling faster and scalable uncertainty quantification.

Findings

Predictions achieve desired coverage levels.

Training and prediction are 2-9 times faster than existing implementations.

Supports multiple conformal methods for regression and classification.

Abstract

Conformal prediction provides a framework for uncertainty quantification, specifically in the forms of prediction intervals and sets with distribution-free guaranteed coverage. While recent cross-conformal techniques such as CV+ and Jackknife+-after-bootstrap achieve better data efficiency than traditional split conformal methods, they incur substantial computational costs due to required pairwise comparisons between training and test samples' out-of-bag scores. Observing that these methods naturally extend from ensemble models, particularly random forests, we leverage existing optimized random forest implementations to enable efficient cross-conformal predictions. We present coverforest, a Python package that implements efficient conformal prediction methods specifically optimized for random forests. coverforest supports both regression and classification tasks through various conformal prediction methods, including split conformal, CV+, Jackknife+-after-bootstrap, and adaptive prediction sets. Our package leverages parallel computing and Cython optimizations to speed up out-of-bag calculations. Our experiments demonstrate that coverforest's predictions achieve the desired level of coverage. In addition, its training and prediction times can be faster than an existing implementation by 2--9 times. The source code for the coverforest is hosted on GitHub at https://github.com/donlap/coverforest.