Understanding an Acquisition Function Family for Bayesian Optimization

TL;DR

This paper analyzes a broad family of Bayesian optimization algorithms to understand their strengths and weaknesses, providing guidance on selecting the most effective approach for different problem types.

Contribution

It offers a comprehensive analysis of a family of acquisition functions in Bayesian optimization, explaining their performance variations across different problem classes.

Findings

Identifies conditions where certain acquisition functions excel

Provides guidelines for choosing acquisition functions based on problem characteristics

Enhances understanding of trade-offs within the acquisition function family

Abstract

Bayesian optimization (BO) developed as an approach for the efficient optimization of expensive black-box functions without gradient information. A typical BO paper introduces a new approach and compares it to some alternatives on simulated and possibly real examples to show its efficacy. Yet on a different example, this new algorithm might not be as effective as the alternatives. This paper looks at a broader family of approaches to explain the strengths and weaknesses of algorithms in the family, with guidance on what choices might work best on different classes of problems.