# A Latent Variational Framework for Stochastic Optimization

## TL;DR

This paper introduces a unifying latent variational framework for stochastic optimization, linking it to Bayesian inference and FBSDEs, which encompasses many existing adaptive gradient methods.

## Contribution

It presents a novel theoretical framework connecting stochastic optimization algorithms with Bayesian inference and stochastic control, unifying various methods under one approach.

## Key findings

- Framework recovers existing adaptive stochastic gradient methods.
- Establishes a connection between optimization algorithms and Bayesian inference.
- Uses FBSDEs to analyze and derive stochastic optimization procedures.

## Abstract

This paper provides a unifying theoretical framework for stochastic optimization algorithms by means of a latent stochastic variational problem. Using techniques from stochastic control, the solution to the variational problem is shown to be equivalent to that of a Forward Backward Stochastic Differential Equation (FBSDE). By solving these equations, we recover a variety of existing adaptive stochastic gradient descent methods. This framework establishes a direct connection between stochastic optimization algorithms and a secondary Bayesian inference problem on gradients, where a prior measure on noisy gradient observations determines the resulting algorithm.