Trust Region-Based Bayesian Optimisation to Discover Diverse Solutions

TL;DR

This paper introduces trust region-based Bayesian optimisation methods to efficiently discover diverse solutions in high-dimensional black-box problems, extending existing algorithms to improve scalability and diversity.

Contribution

It proposes divTuRBO1, an extension of TuRBO1, and two novel approaches for diversity optimisation, demonstrating improved performance in larger dimensions.

Findings

Proposed methods outperform baseline in high dimensions.

Effective in limited evaluation budgets.

Maintain diversity while optimizing for solutions.

Abstract

Bayesian optimisation (BO) is a surrogate-based optimisation technique that efficiently solves expensive black-box functions with small evaluation budgets. Recent studies consider trust regions to improve the scalability of BO approaches when the problem space scales to more dimensions. Motivated by this research, we explore the effectiveness of trust region-based BO algorithms for diversity optimisation in different dimensional black box problems. We propose diversity optimisation approaches extending TuRBO1, which is the first BO method that uses a trust region-based approach for scalability. We extend TuRBO1 as divTuRBO1, which finds an optimal solution while maintaining a given distance threshold relative to a reference solution set. We propose two approaches to find diverse solutions for black-box functions by combining divTuRBO1 runs in a sequential and an interleaving fashion. We conduct experimental investigations on the proposed algorithms and compare their performance with that of the baseline method, ROBOT (rank-ordered Bayesian optimisation with trust regions). We evaluate proposed algorithms on benchmark functions with dimensions 2 to 20. Experimental investigations demonstrate that the proposed methods perform well, particularly in larger dimensions, even with a limited evaluation budget.