On the Nature of Regularity Assumptions in Bilevel Optimization with Constrained Lower-level Problem

TL;DR

This paper examines the regularity assumptions in bilevel optimization, showing that requiring conditions to hold at every upper-level variable is overly strong and rarely true, while holding at almost every $x$ is more prevalent and practical.

Contribution

The paper proves that the strong 'every $x$' regularity condition is non-prevalent and introduces invariants, whereas the weaker 'almost every $x$' condition is prevalent, impacting theory and computation.

Findings

The 'every $x$' condition is non-prevalent and structurally rigid.

The 'almost every $x$' condition is prevalent under random perturbations.

Differences between the two conditions cause fundamental challenges in bilevel optimization.

Abstract

In this paper, we study the regularity assumptions commonly adopted in bilevel optimization with constrained lower-level problems, including the linear independence constraint qualification, the strict complementary slackness condition, and the second-order sufficient condition. These conditions are typically required to hold for the lower-level problem at every upper-level variable $x$. We first show that the requirement that these conditions hold at every upper-level variable $x$ is strong, in the sense that it is non-prevalent: there exist problems for which no sufficiently small perturbation of the lower-level objective and constraints can make the conditions hold at every $x$. To establish the result, we prove rigidity theorems showing that certain structural quantities of the lower-level problem must remain invariant across all $x$ whenever these conditions hold everywhere. We then construct explicit counterexamples in which these invariants differ between two values of $x$. In contrast, we show that the weaker requirement, that these conditions hold at almost every $x$, is a weak assumption, in the sense that it is prevalent: with probability one over a random perturbation of the lower-level objective and constraints, each condition holds at almost every $x$. We further analyze the gap between the two requirements. Although the ``every $x$'' and ``almost every $x$'' versions differ only on a measure-zero set, we show that this difference introduces fundamental difficulties in both theory and computation for bilevel optimization.