Set Smoothness Unlocks Clarke Hyper-stationarity in Bilevel Optimization

TL;DR

This paper introduces set smoothness as a structural property that enables the computation of Clarke hyper-stationary points in nonsmooth bilevel optimization problems, overcoming previous limitations.

Contribution

It establishes the set smoothness property for a broad class of BLO problems and provides the first computational guarantees for Clarke hyper-stationarity in nonsmooth settings.

Findings

Set smoothness holds for many BLO problems.

Weak convexity and concavity of hyper-objectives are ensured.

A zeroth-order algorithm converges to Clarke hyper-stationary points.

Abstract

Solving bilevel optimization (BLO) problems to global optimality is generally intractable. A common surrogate is to compute a hyper-stationary point -- a stationary point of the hyper-objective function obtained by minimizing or maximizing the upper-level objective over the lower-level solution set. Existing methods, however, either provide weak notions of stationarity or require restrictive assumptions to guarantee the smoothness of hyper-objective functions. In this paper, we eliminate these impractical assumptions and show that strong (Clarke) hyper-stationarity remains computable even when the hyper-objective is nonsmooth. Our key ingredient is a new structural property, called set smoothness, which captures the variational dependence of the lower-level solution set on the upper-level variable. We prove that this property holds for a broad class of BLO problems and ensures weak convexity (resp. concavity) of pessimistic (resp. optimistic) hyper-objective functions. Building on this foundation, we show that a zeroth-order algorithm that computes approximate Clarke hyper-stationary points with non-asymptotic convergence guarantees. To the best of our knowledge, this is the first computational guarantee for Clarke-type stationarity in nonsmooth BLO. Beyond this specific application, the set smoothness property emerges as a structural concept of independent interest, with potential to inform the analysis of broader classes of optimization and variational problems.