An Improved Unconstrained Approach for Bilevel Optimization

TL;DR

This paper introduces a novel unconstrained reformulation for nonconvex-strongly-convex bilevel optimization problems, enabling the application of efficient unconstrained algorithms with theoretical guarantees.

Contribution

It transforms bilevel problems into unconstrained ones via a Riemannian manifold approach, allowing existing algorithms to be adapted and analyzed for this class.

Findings

BLO feasible region forms a Riemannian manifold

BLO is equivalent to an unconstrained problem CDB

Existing algorithms can be interpreted as descent methods for CDB

Abstract

In this paper, we focus on the nonconvex-strongly-convex bilevel optimization problem (BLO). In this BLO, the objective function of the upper-level problem is nonconvex and possibly nonsmooth, and the lower-level problem is smooth and strongly convex with respect to the underlying variable $y$. We show that the feasible region of BLO is a Riemannian manifold. Then we transform BLO to its corresponding unconstrained constraint dissolving problem (CDB), whose objective function is explicitly formulated from the objective functions in BLO. We prove that BLO is equivalent to the unconstrained optimization problem CDB. Therefore, various efficient unconstrained approaches, together with their theoretical results, can be directly applied to BLO through CDB. We propose a unified framework for developing subgradient-based methods for CDB. Remarkably, we show that several existing efficient algorithms can fit the unified framework and be interpreted as descent algorithms for CDB. These examples further demonstrate the great potential of our proposed approach.