On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation

TL;DR

This paper develops first-order algorithms for nonconvex bilevel optimization using penalty methods, establishing theoretical connections and convergence guarantees for solving complex hierarchical problems.

Contribution

It introduces a novel penalty-based framework for nonconvex bilevel optimization and provides convergence analysis for first-order stochastic algorithms under minimal assumptions.

Findings

Established conditions linking penalty function and hyper-objective values and derivatives.

Proposed algorithms achieve $ ext{O}(rac{1}{ ext{epsilon}^3})$ and $ ext{O}(rac{1}{ ext{epsilon}^7})$ complexity for deterministic and stochastic cases.

Algorithms can be implemented in a single-loop manner with improved oracle complexity under certain conditions.

Abstract

In this work, we study first-order algorithms for solving Bilevel Optimization (BO) where the objective functions are smooth but possibly nonconvex in both levels and the variables are restricted to closed convex sets. As a first step, we study the landscape of BO through the lens of penalty methods, in which the upper- and lower-level objectives are combined in a weighted sum with penalty parameter $\sigma > 0$. In particular, we establish a strong connection between the penalty function and the hyper-objective by explicitly characterizing the conditions under which the values and derivatives of the two must be $O(\sigma)$-close. A by-product of our analysis is the explicit formula for the gradient of hyper-objective when the lower-level problem has multiple solutions under minimal conditions, which could be of independent interest. Next, viewing the penalty formulation as $O(\sigma)$-approximation of the original BO, we propose first-order algorithms that find an $\epsilon$-stationary solution by optimizing the penalty formulation with $\sigma = O(\epsilon)$. When the perturbed lower-level problem uniformly satisfies the small-error proximal error-bound (EB) condition, we propose a first-order algorithm that converges to an $\epsilon$-stationary point of the penalty function, using in total $O(\epsilon^{-3})$ and $O(\epsilon^{-7})$ accesses to first-order (stochastic) gradient oracles when the oracle is deterministic and oracles are noisy, respectively. Under an additional assumption on stochastic oracles, we show that the algorithm can be implemented in a fully {\it single-loop} manner, i.e., with $O(1)$ samples per iteration, and achieves the improved oracle-complexity of $O(\epsilon^{-3})$ and $O(\epsilon^{-5})$, respectively.