Adaptive Sampling to Reduce Epistemic Uncertainty Using Prediction Interval-Generation Neural Networks

TL;DR

This paper introduces an adaptive sampling method that uses neural network prediction intervals and Gaussian processes to efficiently reduce epistemic uncertainty in predictive models, demonstrated on synthetic and real-world datasets.

Contribution

The paper develops a novel metric for estimating epistemic uncertainty and a batch sampling strategy using GPs, improving convergence speed over existing methods.

Findings

Method converges faster to low epistemic uncertainty levels.

Effective on synthetic and real-world datasets.

Outperforms Normalizing Flows Ensembles, MC-Dropout, and simple GPs.

Abstract

Obtaining high certainty in predictive models is crucial for making informed and trustworthy decisions in many scientific and engineering domains. However, extensive experimentation required for model accuracy can be both costly and time-consuming. This paper presents an adaptive sampling approach designed to reduce epistemic uncertainty in predictive models. Our primary contribution is the development of a metric that estimates potential epistemic uncertainty leveraging prediction interval-generation neural networks. This estimation relies on the distance between the predicted upper and lower bounds and the observed data at the tested positions and their neighboring points. Our second contribution is the proposal of a batch sampling strategy based on Gaussian processes (GPs). A GP is used as a surrogate model of the networks trained at each iteration of the adaptive sampling process. Using this GP, we design an acquisition function that selects a combination of sampling locations to maximize the reduction of epistemic uncertainty across the domain. We test our approach on three unidimensional synthetic problems and a multi-dimensional dataset based on an agricultural field for selecting experimental fertilizer rates. The results demonstrate that our method consistently converges faster to minimum epistemic uncertainty levels compared to Normalizing Flows Ensembles, MC-Dropout, and simple GPs.