Observation-dependent Bayesian active learning via input-warped Gaussian processes

TL;DR

This paper introduces a novel input-warped Gaussian process approach for Bayesian active learning, enabling adaptive exploration of input space based on observed variability, which enhances sample efficiency especially in non-stationary settings.

Contribution

It proposes a new input warping technique with a self-supervised training objective to improve uncertainty quantification in Gaussian process-based active learning.

Findings

Enhanced sample efficiency on active learning benchmarks

Effective handling of non-stationarity in input data

Superior performance over traditional methods

Abstract

Bayesian active learning relies on the precise quantification of predictive uncertainty to explore unknown function landscapes. While Gaussian process surrogates are the standard for such tasks, an underappreciated fact is that their posterior variance depends on the observed outputs only through the hyperparameters, rendering exploration largely insensitive to the actual measurements. We propose to inject observation-dependent feedback by warping the input space with a learned, monotone reparameterization. This mechanism allows the design policy to expand or compress regions of the input space in response to observed variability, thereby shaping the behavior of variance-based acquisition functions. We demonstrate that while such warps can be trained via marginal likelihood, a novel self-supervised objective yields substantially better performance. Our approach improves sample efficiency across a range of active learning benchmarks, particularly in regimes where non-stationarity challenges traditional methods.