Bilevel gradient methods and the Morse parametric qualification condition

TL;DR

This paper introduces the Morse parametric qualification condition for bilevel programming, analyzing gradient algorithms with different strategies in a semi-algebraic setting, bridging strongly convex and generic lower levels.

Contribution

It defines the Morse parametric qualification condition and studies bilevel gradient algorithms with two strategies, highlighting their properties and applications.

Findings

The single-step multi-step strategy is a biased gradient method with rich properties.

The differentiable programming strategy is simpler, less stable, and inspired by meta-learning.

Morse parametric functions form an intermediate class in bilevel programming.

Abstract

We introduce the Morse parametric qualification condition for bilevel programming. Generic semi-algebraic functions are Morse parametric in a piecewise sense. Thus, bilevel programs with a Morse parametric lower level constitute a relevant intermediate class between strongly convex and fully generic lower levels. In this framework, we study bilevel gradient algorithms with two strategies: the single-step multi-step strategy, which involves a sequence of steps on the lower-level problems followed by one step on the upper-level problem, and a differentiable programming strategy that optimizes a smooth approximation of the bilevel problem. While the first is shown to be a biased gradient method on the problem with rich properties, the second, inspired by meta-learning applications, is less stable but offers simplicity and ease of implementation.