A Barrier-Metric First-Order Method for Linearly Constrained Bilevel Optimization

TL;DR

This paper introduces a barrier-metric first-order method for linearly constrained bilevel optimization, addressing nonsmoothness and computational challenges with a barrier smoothing and local geometry analysis.

Contribution

It develops a novel barrier-smoothed surrogate and barrier-aware schedules, enabling efficient gradient-based solutions with provable convergence rates.

Findings

Achieves stationarity rates of  O(K^{-2/3}) in deterministic settings.

Achieves stationarity rates of  O(K^{-2/5}) under stochastic noise.

Provides quantitative bias control as the barrier parameter decreases.

Abstract

We study bilevel optimization with a fixed polyhedral lower feasible set. Such problems are challenging for two reasons: active-set changes can make the upper objective nonsmooth, and existing hypergradient methods typically require lower-Hessian inversions or equivalent linear solves, which are computationally expensive. To address these issues, we adopt a logarithmic barrier smoothing of the lower problem to obtain a differentiable approximation of the constrained bilevel objective, and develop a proxy-gradient algorithm for the resulting barrier-smoothed surrogate. The algorithm uses only gradients of the upper and lower objectives; its only second-order object is the explicit logarithmic barrier Hessian determined by the fixed polyhedral constraints. Barrier smoothing restores differentiability, but Euclidean smoothness constants are not uniformly bounded near the boundary. We therefore develop a local Dikin-geometry analysis in which the barrier-metric provides an oracle-free curvature scale near the moving lower centers. This leads to barrier-aware schedules that keep the iterates inside locally well-behaved regions. For the barrier-smoothed objective, we prove stationarity rates of $\widetilde{O}(K^{-2/3})$ in the deterministic setting and $\widetilde{O}(K^{-2/5})$ under upper-level-only bounded stochastic noise after $K$ outer iterations, together with quantitative bias control as the barrier parameter decreases.