Bayesian Optimization of Bilevel Problems

TL;DR

This paper introduces a Bayesian Optimization approach for complex bilevel problems with black-box functions, leveraging Gaussian processes and a novel acquisition function to improve sample efficiency and solution quality.

Contribution

It presents a new Bayesian Optimization framework for bilevel problems with black-box functions, including a novel acquisition function and knowledge transfer mechanism.

Findings

The proposed method is highly sample-efficient.

It outperforms existing methods in solution quality.

Experimental results validate the effectiveness of the approach.

Abstract

Bilevel optimization, a hierarchical mathematical framework where one optimization problem is nested within another, has emerged as a powerful tool for modeling complex decision-making processes in various fields such as economics, engineering, and machine learning. This paper focuses on bilevel optimization where both upper-level and lower-level functions are black boxes and expensive to evaluate. We propose a Bayesian Optimization framework that models the upper and lower-level functions as Gaussian processes over the combined space of upper and lower-level decisions, allowing us to exploit knowledge transfer between different sub-problems. Additionally, we propose a novel acquisition function for this model. Our experimental results demonstrate that the proposed algorithm is highly sample-efficient and outperforms existing methods in finding high-quality solutions.