Bayesian optimization for computationally extensive probability distributions

TL;DR

This paper introduces a Bayesian optimization method that efficiently finds better maximizers of computationally intensive probability distributions, outperforming traditional methods like random search and Monte Carlo, especially in physical model estimation.

Contribution

The paper proposes a novel Bayesian optimization approach that uses extreme values of acquisition functions to locate maxima in computationally extensive probability distributions, demonstrating improved performance.

Findings

Outperforms random search, steepest descent, and Monte Carlo methods.

Effective in physical model estimation with limited sampling points.

Combines Bayesian optimization with steepest descent for enhanced results.

Abstract

An efficient method for finding a better maximizer of computationally extensive probability distributions is proposed on the basis of a Bayesian optimization technique. A key idea of the proposed method is to use extreme values of acquisition functions by Gaussian processes for the next training phase, which should be located near a local maximum or a global maximum of the probability distribution. Our Bayesian optimization technique is applied to the posterior distribution in the effective physical model estimation, which is a computationally extensive probability distribution. Even when the number of sampling points on the posterior distributions is fixed to be small, the Bayesian optimization provides a better maximizer of the posterior distributions in comparison to those by the random search method, the steepest descent method, or the Monte Carlo method. Furthermore, the Bayesian optimization improves the results efficiently by combining the steepest descent method and thus it is a powerful tool to search for a better maximizer of computationally extensive probability distributions.