Safe Gradient Flow for Bilevel Optimization

TL;DR

This paper introduces a control-theoretic safe gradient flow method for bilevel optimization, combining convergence guarantees with practical scalability improvements, validated through numerical experiments.

Contribution

It proposes a novel safe gradient flow approach with a relaxed formulation for scalable bilevel optimization, including theoretical convergence analysis and practical validation.

Findings

Converges to a neighborhood of the optimal solution.

Ensures lower-level decision variables stay within suboptimality bounds.

Validated effectiveness through numerical experiments.

Abstract

Bilevel optimization is a key framework in hierarchical decision-making, where one problem is embedded within the constraints of another. In this work, we propose a control-theoretic approach to solving bilevel optimization problems. Our method consists of two components: a gradient flow mechanism to minimize the upper-level objective and a safety filter to enforce the constraints imposed by the lower-level problem. Together, these components form a safe gradient flow that solves the bilevel problem in a single loop. To improve scalability with respect to the lower-level problem's dimensions, we introduce a relaxed formulation and design a compact variant of the safe gradient flow. This variant minimizes the upper-level objective while ensuring the lower-level decision variable remains within a user-defined suboptimality. Using Lyapunov analysis, we establish convergence guarantees for the dynamics, proving that they converge to a neighborhood of the optimal solution. Numerical experiments further validate the effectiveness of the proposed approaches. Our contributions provide both theoretical insights and practical tools for efficiently solving bilevel optimization problems.