A Review of Bilevel Optimization: Methods, Emerging Applications, and Recent Advancements

TL;DR

This paper provides a comprehensive review of bilevel optimization techniques, their applications in various fields, and recent advancements including decomposition methods and neural architecture search in machine learning.

Contribution

It offers an extensive overview of classical and evolutionary methods for solving bilevel problems and discusses emerging applications like neural architecture search.

Findings

Various classical and evolutionary approaches are used for solving bilevel problems.

Bilevel optimization is applied in hierarchical decision-making and machine learning.

Recent advancements include decomposition methods and neural architecture search applications.

Abstract

This paper presents a comprehensive review of techniques proposed in the literature for solving bilevel optimization problems encountered in various real-life applications. Bilevel optimization is an appropriate choice for hierarchical decision-making situations, where a decision-maker needs to consider a possible response from stakeholder(s) for each of its actions to achieve his own goals. Mathematically, it leads to a nested optimization structure, in which a primary (leader's) optimization problem contains a secondary (follower's) optimization problem as a constraint. Various forms of bilevel problems, including linear, mixed-integer, single-objective, and multi-objective, are covered. For bilevel problem solving methods, various classical and evolutionary approaches are explained. Along with an overview of various areas of applications, two recent considerations of bilevel approach are introduced. The first application involves a bilevel decomposition approach for solving general optimization problems, and the second application involves Neural Architecture Search (NAS), which is a prime example of a bilevel optimization problem in the area of machine learning.