Expensive Multi-Objective Bayesian Optimization Based on Diffusion Models

TL;DR

This paper introduces CDM-PSL, a novel diffusion model-based approach for multi-objective Bayesian optimization that effectively models complex Pareto distributions and balances multiple objectives, outperforming existing methods on benchmarks.

Contribution

The paper proposes a new diffusion model-based Pareto set learning algorithm for expensive MOBO, incorporating an entropy-based weighting and guiding strategy to improve stability and performance.

Findings

Outperforms state-of-the-art MOBO algorithms on benchmarks

Effectively models complex Pareto distributions with diffusion models

Balances multiple objectives using entropy-based weighting

Abstract

Multi-objective Bayesian optimization (MOBO) has shown promising performance on various expensive multi-objective optimization problems (EMOPs). However, effectively modeling complex distributions of the Pareto optimal solutions is difficult with limited function evaluations. Existing Pareto set learning algorithms may exhibit considerable instability in such expensive scenarios, leading to significant deviations between the obtained solution set and the Pareto set (PS). In this paper, we propose a novel Composite Diffusion Model based Pareto Set Learning algorithm, namely CDM-PSL, for expensive MOBO. CDM-PSL includes both unconditional and conditional diffusion model for generating high-quality samples. Besides, we introduce an information entropy based weighting method to balance different objectives of EMOPs. This method is integrated with the guiding strategy, ensuring that all the objectives are appropriately balanced and given due consideration during the optimization process; Extensive experimental results on both synthetic benchmarks and real-world problems demonstrates that our proposed algorithm attains superior performance compared with various state-of-the-art MOBO algorithms.