Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation

TL;DR

This paper introduces PLAyBOOK, an asynchronous batch Bayesian optimization method that improves resource utilization and often outperforms synchronous approaches in hyperparameter tuning tasks.

Contribution

The paper proposes PLAyBOOK, a novel asynchronous BO algorithm with local penalization, addressing resource waste in parallel hyperparameter optimization.

Findings

PLAyBOOK outperforms synchronous BO in wall-clock time

Asynchronous BO reduces idle worker time

Empirical validation on synthetic and real-world tasks

Abstract

Batch Bayesian optimisation (BO) has been successfully applied to hyperparameter tuning using parallel computing, but it is wasteful of resources: workers that complete jobs ahead of others are left idle. We address this problem by developing an approach, Penalising Locally for Asynchronous Bayesian Optimisation on $k$ workers (PLAyBOOK), for asynchronous parallel BO. We demonstrate empirically the efficacy of PLAyBOOK and its variants on synthetic tasks and a real-world problem. We undertake a comparison between synchronous and asynchronous BO, and show that asynchronous BO often outperforms synchronous batch BO in both wall-clock time and number of function evaluations.