Functionally Constrained Algorithm Solves Convex Simple Bilevel Problems

TL;DR

This paper introduces a novel algorithm for simple convex bilevel problems, reformulating them as functionally constrained problems, achieving near-optimal rates under standard smoothness or Lipschitz assumptions.

Contribution

The paper presents the first near-optimal algorithm for convex simple bilevel problems that works under standard smoothness or Lipschitz conditions.

Findings

Achieves near-optimal convergence rates for smooth and nonsmooth problems.

Demonstrates fundamental difficulty of simple bilevel problems for first-order zero-respecting algorithms.

Reformulates bilevel problems into functionally constrained problems for improved solvability.

Abstract

This paper studies simple bilevel problems, where a convex upper-level function is minimized over the optimal solutions of a convex lower-level problem. We first show the fundamental difficulty of simple bilevel problems, that the approximate optimal value of such problems is not obtainable by first-order zero-respecting algorithms. Then we follow recent works to pursue the weak approximate solutions. For this goal, we propose a novel method by reformulating them into functionally constrained problems. Our method achieves near-optimal rates for both smooth and nonsmooth problems. To the best of our knowledge, this is the first near-optimal algorithm that works under standard assumptions of smoothness or Lipschitz continuity for the objective functions.