# Variable Metric Stochastic Approximation Theory

## TL;DR

This paper develops a convergence theory for variable metric stochastic approximation methods, including online BFGS and LBFGS algorithms, with implications for learning from expert advice.

## Contribution

It introduces a comprehensive convergence framework for a broad class of online variable metric algorithms, extending existing methods like BFGS and LBFGS.

## Key findings

- Convergence guarantees for online BFGS and LBFGS algorithms.
- Applicability of the theory to learning from expert advice.
- Enhanced understanding of variable metric stochastic approximation methods.

## Abstract

We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results for learning from expert advice.