Efficient Epistemic Uncertainty Estimation in Regression Ensemble Models Using Pairwise-Distance Estimators

TL;DR

This paper presents PaiDEs, a fast and scalable pairwise-distance estimator for epistemic uncertainty in ensemble regression models, significantly improving active learning performance especially in high-dimensional settings.

Contribution

Introduces PaiDEs, a novel pairwise-distance based method for efficient epistemic uncertainty estimation in ensemble regression models, outperforming existing methods in speed and accuracy.

Findings

PaiDEs estimate uncertainty up to 100 times faster than Monte Carlo methods.

PaiDEs outperform existing active learning methods in high-dimensional regression tasks.

The approach demonstrates superior performance on multiple benchmark datasets.

Abstract

This work introduces an efficient novel approach for epistemic uncertainty estimation for ensemble models for regression tasks using pairwise-distance estimators (PaiDEs). Utilizing the pairwise-distance between model components, these estimators establish bounds on entropy. We leverage this capability to enhance the performance of Bayesian Active Learning by Disagreement (BALD). Notably, unlike sample-based Monte Carlo estimators, PaiDEs exhibit a remarkable capability to estimate epistemic uncertainty at speeds up to 100 times faster while covering a significantly larger number of inputs at once and demonstrating superior performance in higher dimensions. To validate our approach, we conducted a varied series of regression experiments on commonly used benchmarks: 1D sinusoidal data, $\textit{Pendulum}$, $\textit{Hopper}$, $\textit{Ant}$ and $\textit{Humanoid}$. For each experimental setting, an active learning framework was applied to demonstrate the advantages of PaiDEs for epistemic uncertainty estimation. We compare our approach to existing active learning methods and find that our approach outperforms on high-dimensional regression tasks.