Generative Multi-Form Bayesian Optimization

TL;DR

This paper introduces GMFoO, a multi-form Bayesian optimization method that optimizes over multiple correlated latent spaces simultaneously, improving convergence and solution quality for complex structured input problems.

Contribution

It proposes a novel multi-form GMO approach with correlated latent spaces and information exchange strategies, enhancing optimization over complex structured inputs.

Findings

GMFoO outperforms single-form methods in convergence speed.

It achieves better solutions within limited computational budgets.

Experimental results on airfoil, corbel, and area maximization problems confirm its effectiveness.

Abstract

Many real-world problems, such as airfoil design, involve optimizing a black-box expensive objective function over complex structured input space (e.g., discrete space or non-Euclidean space). By mapping the complex structured input space into a latent space of dozens of variables, a two-stage procedure labeled as generative model based optimization (GMO) in this paper, shows promise in solving such problems. However, the latent dimension of GMO is hard to determine, which may trigger the conflicting issue between desirable solution accuracy and convergence rate. To address the above issue, we propose a multi-form GMO approach, namely generative multi-form optimization (GMFoO), which conducts optimization over multiple latent spaces simultaneously to complement each other. More specifically, we devise a generative model which promotes positive correlation between latent spaces to facilitate effective knowledge transfer in GMFoO. And further, by using Bayesian optimization (BO) as the optimizer, we propose two strategies to exchange information between these latent spaces continuously. Experimental results are presented on airfoil and corbel design problems and an area maximization problem as well to demonstrate that our proposed GMFoO converges to better designs on a limited computational budget.