Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weights

TL;DR

This paper introduces a Gaussian Process-based active learning method for modeling functions under unknown Boltzmann distributions, addressing challenges in computational chemistry and related fields.

Contribution

It proposes B-SID-iVAR, an acquisition function that approximates intractable distributions without partition function estimation, applicable to discrete and continuous domains.

Findings

Method outperforms existing approaches on synthetic benchmarks.

Achieves vanishing prediction error with high probability.

Demonstrates effectiveness in PES modeling and drug discovery tasks.

Abstract

We consider the active learning problem where the goal is to learn an unknown function with low prediction error under an unknown Boltzmann distribution induced by the function itself. This self-induced weighting arises naturally in problems such as potential energy surface (PES) modeling in computational chemistry, yet poses unique challenges as the target distribution is unknown and its partition function is intractable. We propose \texttt{AB-SID-iVAR}, a Gaussian Process-based acquisition function that approximates the intractable Bayesian target distribution in closed form while avoiding partition function estimation, and is applicable to both discrete and continuous input domains. We also analyze a Thompson sampling alternative (\texttt{TS-SID-iVAR}) as a higher variance Monte Carlo variant. Despite the unknown target, under mild conditions, we establish that the terminal prediction error vanishes with high probability, and provide a tighter average-case guarantee. We demonstrate consistent improvements over existing approaches in this setting on synthetic benchmarks and real-world PES modeling and drug discovery tasks.