Near-Optimal Algorithms for Convex Simple Bilevel Optimization under Weak Assumptions

TL;DR

This paper introduces near-optimal algorithms for simple bilevel convex optimization, achieving near-matching lower bounds under mild assumptions by leveraging a novel dual approach and root-finding methods.

Contribution

It presents a new dual approach and root-finding framework that attain near-optimal complexity for convex simple bilevel problems under weak assumptions.

Findings

Achieves near-optimal complexity of ( ilde{ ext{ extasciitilde}}rac{1}{\u03b5})

Aligns with lower bounds for smooth convex bilevel problems

Uses a novel dual approach and root-finding techniques

Abstract

This paper considers the simple bilevel optimization (SBO) problem, which minimizes a composite convex function over the optimal solution set of another composite convex minimization problem. We first show that this bilevel problem is equivalent to finding the left-most root of a nonlinear equation. Based on this and a novel dual approach for solving the subproblem in each iteration, we efficiently obtain an $(\epsilon, \epsilon)$-optimal solution through the bisection and Newton methods. The proposed methods achieve near-optimal operation complexity of ${\tilde{\mathcal{O}}(\sqrt{1/\epsilon})}$ under mild assumptions, aligning with the lower complexity bounds of the first-order methods in SBO with both level objectives being smooth convex and unconstrained composite convex optimization when ignoring logarithmic terms.