Optimality Conditions and Numerical Algorithms for a Class of Minimax Bilevel Optimization Problems

TL;DR

This paper introduces optimality conditions and develops efficient algorithms for a novel class of minimax bilevel optimization problems, with applications in game theory, machine learning, and power systems.

Contribution

It establishes the first optimality conditions for minimax bilevel problems and proposes a penalty-based first-order algorithm with accelerated convergence.

Findings

Algorithms find an $ta$-KKT solution within $\u2206(ta^{-3}\u2206(\u001eta^{-1}))$ iterations.

Nesterov accelerated extension improves convergence rate.

Numerical experiments demonstrate effectiveness in various applications.

Abstract

In many applications, including Stackelberg games, machine learning, and power systems \cite{Mackay2018Selftuning,Heinrich1952The,Wang2021Bi-Level}, the decisions in a minimax optimization problem can be constrained by a solution to an optimization problem. In this paper, we introduce optimality conditions of this novel minimax bilevel optimization problem and develops efficient first-order algorithms for this class of problems. Firstly, we establish the optimality conditions for minimax bilevel problems by reconstructing the lower-level problem through its Karush-Kuhn-Tucker (KKT) conditions and value function. Secondly, we develop a penalty method framework to approximately solve the minimax bilevel problem by transforming it into a single-level minimax problem. Thirdly, we design a projected gradient multi-step ascent descent method to solve the resulting minimax problem, which can find an $\epsilon$-KKT solution for the original minimax bilevel problem within $\mathcal{O}(\epsilon^{-3} \log(\epsilon^{-1}))$ iterations. To improve {the convergence rate} of the algorithm, we provide its Nesterov accelerated extension with $\mathcal{O}(\epsilon^{-3} \log(\epsilon^{-1}))$ iteration complexity. Finally, we demonstrate the effectiveness of our model and algorithms through numerical experiments on various minimax bilevel optimization problems and a bilevel economic dispatch in the power system.