Reliable Prediction Intervals with Directly Optimized Inductive Conformal Regression for Deep Learning

TL;DR

This paper introduces DOICR, a novel method for generating prediction intervals in deep learning regression that directly optimizes for minimal width while maintaining coverage, outperforming existing methods.

Contribution

The paper proposes DOICR, a new approach that directly optimizes the width of prediction intervals under coverage constraints, improving over traditional ICP methods.

Findings

DOICR achieves narrower prediction intervals with guaranteed coverage.

Benchmark results show DOICR outperforms state-of-the-art algorithms.

Effective for both tabular and image data in deep learning regression.

Abstract

By generating prediction intervals (PIs) to quantify the uncertainty of each prediction in deep learning regression, the risk of wrong predictions can be effectively controlled. High-quality PIs need to be as narrow as possible, whilst covering a preset proportion of real labels. At present, many approaches to improve the quality of PIs can effectively reduce the width of PIs, but they do not ensure that enough real labels are captured. Inductive Conformal Predictor (ICP) is an algorithm that can generate effective PIs which is theoretically guaranteed to cover a preset proportion of data. However, typically ICP is not directly optimized to yield minimal PI width. However, in this study, we use Directly Optimized Inductive Conformal Regression (DOICR) that takes only the average width of PIs as the loss function and increases the quality of PIs through an optimized scheme under the validity condition that sufficient real labels are captured in the PIs. Benchmark experiments show that DOICR outperforms current state-of-the-art algorithms for regression problems using underlying Deep Neural Network structures for both tabular and image data.