Confidence Intervals and Simultaneous Confidence Bands Based on Deep Learning

TL;DR

This paper introduces a non-parametric bootstrap method for constructing valid confidence intervals and bands for deep learning predictions, especially for survival data with right censoring, addressing a key gap in uncertainty quantification.

Contribution

It proposes a simple, adaptable bootstrap approach that accurately quantifies uncertainty in deep neural network predictions without disrupting training.

Findings

The method produces valid, non-overly conservative confidence intervals.

It can be integrated into any deep neural network architecture.

Demonstrated effectiveness on survival data with right censoring.

Abstract

Deep learning models have significantly improved prediction accuracy in various fields, gaining recognition across numerous disciplines. Yet, an aspect of deep learning that remains insufficiently addressed is the assessment of prediction uncertainty. Producing reliable uncertainty estimators could be crucial in practical terms. For instance, predictions associated with a high degree of uncertainty could be sent for further evaluation. Recent works in uncertainty quantification of deep learning predictions, including Bayesian posterior credible intervals and a frequentist confidence-interval estimation, have proven to yield either invalid or overly conservative intervals. Furthermore, there is currently no method for quantifying uncertainty that can accommodate deep neural networks for survival (time-to-event) data that involves right-censored outcomes. In this work, we provide a valid non-parametric bootstrap method that correctly disentangles data uncertainty from the noise inherent in the adopted optimization algorithm, ensuring that the resulting point-wise confidence intervals or the simultaneous confidence bands are accurate (i.e., valid and not overly conservative). The proposed ad-hoc method can be easily integrated into any deep neural network without interfering with the training process. The utility of the proposed approach is illustrated by constructing simultaneous confidence bands for survival curves derived from deep neural networks for survival data with right censoring.