Online Nonconvex Bilevel Optimization with Bregman Divergences

TL;DR

This paper introduces novel online bilevel optimization algorithms using Bregman divergences, achieving improved regret rates and efficiency for dynamic machine learning tasks like hyperparameter tuning and meta-learning.

Contribution

It presents the first stochastic online bilevel optimizer with variance reduction and a deterministic Bregman-based method that adapts to problem geometry, advancing online bilevel optimization.

Findings

OBBO improves sublinear regret rates with hypergradient error decomposition.

SOBBO achieves sublinear regret and reduces variance without extra gradient samples.

Algorithms outperform existing online and offline bilevel methods in experiments.

Abstract

Bilevel optimization methods are increasingly relevant within machine learning, especially for tasks such as hyperparameter optimization and meta-learning. Compared to the offline setting, online bilevel optimization (OBO) offers a more dynamic framework by accommodating time-varying functions and sequentially arriving data. This study addresses the online nonconvex-strongly convex bilevel optimization problem. In deterministic settings, we introduce a novel online Bregman bilevel optimizer (OBBO) that utilizes adaptive Bregman divergences. We demonstrate that OBBO enhances the known sublinear rates for bilevel local regret through a novel hypergradient error decomposition that adapts to the underlying geometry of the problem. In stochastic contexts, we introduce the first stochastic online bilevel optimizer (SOBBO), which employs a window averaging method for updating outer-level variables using a weighted average of recent stochastic approximations of hypergradients. This approach not only achieves sublinear rates of bilevel local regret but also serves as an effective variance reduction strategy, obviating the need for additional stochastic gradient samples at each timestep. Experiments on online hyperparameter optimization and online meta-learning highlight the superior performance, efficiency, and adaptability of our Bregman-based algorithms compared to established online and offline bilevel benchmarks.