Optimization of Bregman Variational Learning Dynamics

TL;DR

This paper introduces a comprehensive optimization framework for Bregman-Variational Learning Dynamics, unifying various learning methods and providing stability guarantees in nonstationary environments.

Contribution

It formulates a new class of operator-based updates using variational optimization with Bregman divergences, establishing their stability and convergence properties.

Findings

Operators are averaged, contractive, and exponentially stable.

Proved Fejer monotonicity and drift-aware convergence.

Established continuous-time equivalence via EVI.

Abstract

We develop a general optimization-theoretic framework for Bregman-Variational Learning Dynamics (BVLD), a new class of operator-based updates that unify Bayesian inference, mirror descent, and proximal learning under time-varying environments. Each update is formulated as a variational optimization problem combining a smooth convex loss f_t with a Bregman divergence D_psi. We prove that the induced operator is averaged, contractive, and exponentially stable in the Bregman geometry. Further, we establish Fejer monotonicity, drift-aware convergence, and continuous-time equivalence via an evolution variational inequality (EVI). Together, these results provide a rigorous analytical foundation for well-posed and stability-guaranteed operator dynamics in nonstationary optimization.