Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spaces

TL;DR

This paper introduces a graph-based Bayesian optimization method for mixed-variable problems, leveraging graph autoencoders to model variable interactions and improve search efficiency.

Contribution

It proposes a novel framework combining graph structure learning with latent space optimization for mixed-variable Bayesian optimization.

Findings

Outperforms existing methods on synthetic and real-world tasks.

Demonstrates significant computational efficiency.

Provides empirical evidence of graph structures in optimization problems.

Abstract

Real-world optimization problems are generally not just black-box problems, but also involve mixed types of inputs in which discrete and continuous variables coexist. Such mixed-space optimization possesses the primary challenge of modeling complex interactions between the inputs. In this work, we propose a novel yet simple approach that entails exploiting the graph data structure to model the underlying relationship between variables, i.e., variables as nodes and interactions defined by edges. Then, a variational graph autoencoder is used to naturally take the interactions into account. We first provide empirical evidence of the existence of such graph structures and then suggest a joint framework of graph structure learning and latent space optimization to adaptively search for optimal graph connectivity. Experimental results demonstrate that our method shows remarkable performance, exceeding the existing approaches with significant computational efficiency for a number of synthetic and real-world tasks.