Simple and Scalable Parallelized Bayesian Optimization

TL;DR

This paper introduces a simple, scalable Bayesian optimization method designed for asynchronous parallel settings, effectively utilizing distributed resources for high-cost problems like hyperparameter tuning.

Contribution

The paper presents a novel asynchronous parallel Bayesian optimization approach that is simple to implement and scalable, addressing limitations of previous methods.

Findings

Demonstrates promising performance on benchmark functions

Effective hyperparameter optimization for multi-layer perceptrons

Scalable and efficient in asynchronous parallel environments

Abstract

In recent years, leveraging parallel and distributed computational resources has become essential to solve problems of high computational cost. Bayesian optimization (BO) has shown attractive results in those expensive-to-evaluate problems such as hyperparameter optimization of machine learning algorithms. While many parallel BO methods have been developed to search efficiently utilizing these computational resources, these methods assumed synchronous settings or were not scalable. In this paper, we propose a simple and scalable BO method for asynchronous parallel settings. Experiments are carried out with a benchmark function and hyperparameter optimization of multi-layer perceptrons, which demonstrate the promising performance of the proposed method.