Likelihood-ratio-based confidence intervals for neural networks

TL;DR

This paper presents DeepLR, a likelihood-ratio-based method for constructing confidence intervals in neural networks, capable of asymmetric and data-dependent intervals, with potential applications in high-stakes fields despite high computational costs.

Contribution

Introduces DeepLR, a novel likelihood-ratio-based approach for neural network confidence intervals that accounts for training factors and data scarcity.

Findings

DeepLR can produce asymmetric confidence intervals.

The method incorporates training time, architecture, and regularization.

High computational cost limits widespread use but suits critical applications.

Abstract

This paper introduces a first implementation of a novel likelihood-ratio-based approach for constructing confidence intervals for neural networks. Our method, called DeepLR, offers several qualitative advantages: most notably, the ability to construct asymmetric intervals that expand in regions with a limited amount of data, and the inherent incorporation of factors such as the amount of training time, network architecture, and regularization techniques. While acknowledging that the current implementation of the method is prohibitively expensive for many deep-learning applications, the high cost may already be justified in specific fields like medical predictions or astrophysics, where a reliable uncertainty estimate for a single prediction is essential. This work highlights the significant potential of a likelihood-ratio-based uncertainty estimate and establishes a promising avenue for future research.