Bayesian Optimization of Combinatorial Structures

TL;DR

This paper introduces a novel Bayesian optimization algorithm tailored for combinatorial structures, leveraging semidefinite programming to efficiently handle large, complex search spaces with limited data, outperforming existing methods.

Contribution

The paper presents the first scalable Bayesian optimization algorithm for combinatorial structures using semidefinite programming, addressing data scarcity and computational challenges.

Findings

Outperforms existing combinatorial optimization methods

Efficiently handles large search spaces with scarce data

Uses semidefinite programming for scalability

Abstract

The optimization of expensive-to-evaluate black-box functions over combinatorial structures is an ubiquitous task in machine learning, engineering and the natural sciences. The combinatorial explosion of the search space and costly evaluations pose challenges for current techniques in discrete optimization and machine learning, and critically require new algorithmic ideas. This article proposes, to the best of our knowledge, the first algorithm to overcome these challenges, based on an adaptive, scalable model that identifies useful combinatorial structure even when data is scarce. Our acquisition function pioneers the use of semidefinite programming to achieve efficiency and scalability. Experimental evaluations demonstrate that this algorithm consistently outperforms other methods from combinatorial and Bayesian optimization.