Accelerating Diagonal Methods for Bilevel Optimization: Unified Convergence via Continuous-Time Dynamics

TL;DR

This paper provides a unified convergence analysis for accelerated diagonal methods in bilevel optimization using continuous-time dynamics, offering explicit rates and insights into geometric influences.

Contribution

It introduces a unified convergence framework for fast diagonal methods in bilevel problems based on continuous-time dynamics analysis, extending recent results and providing new insights.

Findings

Explicit convergence rates derived

Weak convergence to solutions guaranteed

Numerical experiments confirm theoretical advantages

Abstract

We analyze fast diagonal methods for simple bilevel programs. Guided by the analysis of the corresponding continuous-time dynamics, we provide a unified convergence analysis under general geometric conditions, including H\"olderian growth and the Attouch-Czarnecki condition. Our results yield explicit convergence rates and guarantee weak convergence to a solution of the bilevel problem. In particular, we improve and extend recent results on accelerated schemes, offering novel insights into the trade-offs between geometry, regularization decay, and algorithmic design. Numerical experiments illustrate the advantages of more flexible methods and support our theoretical findings.