Achieving Better Local Regret Bound for Online Non-Convex Bilevel Optimization

TL;DR

This paper establishes optimal regret bounds for online bilevel optimization, proposing algorithms that achieve these bounds with efficient gradient evaluations and validated by experiments.

Contribution

It introduces new algorithms with optimal regret bounds for both standard and window-averaged bilevel local regret in online bilevel optimization.

Findings

Achieved optimal regret $oxed{ ext{Omega}(1+V_T)}$ with $O(T ext{log} T)$ evaluations.

Designed a single-loop algorithm with regret including gradient-variation terms.

Achieved optimal regret $oxed{ ext{Omega}(T/W^2)}$ for window-averaged regret, with efficient single-loop structure.

Abstract

Online bilevel optimization (OBO) has emerged as a powerful framework for many machine learning problems. Prior works have developed several algorithms that minimize the standard bilevel local regret or the window-averaged bilevel local regret of the OBO problem, but the optimality of existing regret bounds remains unclear. In this work, we establish optimal regret bounds for both settings. For standard bilevel local regret, we propose an algorithm with adaptive iteration strategy that achieves the optimal regret $\Omega(1+V_T)$ with at most $O(T\log T)$ total inner-level gradient evaluations. We further develop a fully single-loop algorithm whose regret bound includes an additional gradient-variation terms. For the window-averaged bilevel local regret, we design an algorithm that captures linear environmental variation through a novel window-based analysis and achieves the optimal regret $\Omega(T/W^2)$. The algorithm also supports an efficient single-loop structure, achieving an $O(T/W)$ regret bound with $O(WT)$ total gradient evaluations. Experiments validate our theoretical findings and demonstrate the practical effectiveness of the proposed methods.