A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse

TL;DR

This paper introduces a novel fully single loop bilevel optimization algorithm that avoids Hessian inverse computations, improving efficiency while maintaining convergence guarantees, and demonstrates its effectiveness on machine learning tasks.

Contribution

The paper develops the first fully single loop bilevel optimization algorithm that does not require Hessian inverse, with theoretical convergence analysis and empirical validation.

Findings

Converges with rate O(ε^{-2})

Effectively handles hyper-gradient approximation

Outperforms double loop methods in experiments

Abstract

In this paper, we propose a new Hessian inverse free Fully Single Loop Algorithm (FSLA) for bilevel optimization problems. Classic algorithms for bilevel optimization admit a double loop structure which is computationally expensive. Recently, several single loop algorithms have been proposed with optimizing the inner and outer variable alternatively. However, these algorithms not yet achieve fully single loop. As they overlook the loop needed to evaluate the hyper-gradient for a given inner and outer state. In order to develop a fully single loop algorithm, we first study the structure of the hyper-gradient and identify a general approximation formulation of hyper-gradient computation that encompasses several previous common approaches, e.g. back-propagation through time, conjugate gradient, \emph{etc.} Based on this formulation, we introduce a new state variable to maintain the historical hyper-gradient information. Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm. We theoretically show that the error generated by the new state can be bounded and our algorithm converges with the rate of $O(\epsilon^{-2})$. Finally, we verify the efficacy our algorithm empirically through multiple bilevel optimization based machine learning tasks.