Non-Stationary Functional Bilevel Optimization

TL;DR

SmoothFBO is a novel algorithm for non-stationary functional bilevel optimization that offers theoretical guarantees and outperforms existing methods in online, dynamic environments.

Contribution

It introduces a time-smoothed stochastic hypergradient estimator for non-stationary FBO, extending bilevel optimization to online, non-stationary scenarios with theoretical and practical benefits.

Findings

Outperforms existing FBO methods in non-stationary hyperparameter tuning

Provides theoretical guarantees with sublinear regret

Demonstrates effectiveness in model-based reinforcement learning

Abstract

Functional bilevel optimization (FBO) provides a powerful framework for hierarchical learning in function spaces, yet current methods are limited to static offline settings and perform suboptimally in online, non-stationary scenarios. We propose SmoothFBO, the first algorithm for non-stationary FBO with both theoretical guarantees and practical scalability. SmoothFBO introduces a time-smoothed stochastic hypergradient estimator that reduces variance through a window parameter, enabling stable outer-loop updates with sublinear regret. Importantly, the classical parametric bilevel case is a special reduction of our framework, making SmoothFBO a natural extension to online, non-stationary settings. Empirically, SmoothFBO consistently outperforms existing FBO methods in non-stationary hyperparameter optimization and model-based reinforcement learning, demonstrating its practical effectiveness. Together, these results establish SmoothFBO as a general, theoretically grounded, and practically viable foundation for bilevel optimization in online, non-stationary scenarios.