# A Stochastic Interpretation of Stochastic Mirror Descent: Risk-Sensitive Optimality

## TL;DR

This paper presents a new interpretation of stochastic mirror descent as a risk-sensitive optimal estimator within exponential family distributions, and proposes a modified symmetric version of SMD.

## Contribution

It introduces a risk-sensitive interpretation of SMD and proposes a symmetric variant, extending theoretical understanding of these algorithms in non-Gaussian settings.

## Key findings

- SMD can be viewed as a risk-sensitive estimator for exponential family distributions.
- A modified symmetric SMD (SSMD) is proposed based on this interpretation.
- The analysis extends SMD properties beyond Gaussian assumptions using Bregman divergence.

## Abstract

Stochastic mirror descent (SMD) is a fairly new family of algorithms that has recently found a wide range of applications in optimization, machine learning, and control. It can be considered a generalization of the classical stochastic gradient algorithm (SGD), where instead of updating the weight vector along the negative direction of the stochastic gradient, the update is performed in a "mirror domain" defined by the gradient of a (strictly convex) potential function. This potential function, and the mirror domain it yields, provides considerable flexibility in the algorithm compared to SGD. While many properties of SMD have already been obtained in the literature, in this paper we exhibit a new interpretation of SMD, namely that it is a risk-sensitive optimal estimator when the unknown weight vector and additive noise are non-Gaussian and belong to the exponential family of distributions. The analysis also suggests a modified version of SMD, which we refer to as symmetric SMD (SSMD). The proofs rely on some simple properties of Bregman divergence, which allow us to extend results from quadratics and Gaussians to certain convex functions and exponential families in a rather seamless way.