A General Convergence Result for Mirror Descent with Armijo Line Search

TL;DR

This paper introduces a local relative smoothness condition for mirror descent with Armijo line search, ensuring convergence even when traditional assumptions fail, and demonstrates its effectiveness in quantum state tomography.

Contribution

It generalizes the convergence conditions for mirror descent by introducing a local relative smoothness criterion, broadening its applicability.

Findings

Mirror descent with Armijo line search converges under local relative smoothness.

The method outperforms others in quantum state tomography experiments.

It provides the fastest guaranteed convergence in tested scenarios.

Abstract

Existing convergence guarantees for the mirror descent algorithm require the objective function to have a bounded gradient or be smooth relative to a Legendre function. The bounded gradient and relative smoothness conditions, however, may not hold in important applications, such as quantum state tomography and portfolio selection. In this paper, we propose a local version of the relative smoothness condition as a generalization of its existing global version, and prove that under this local relative smoothness condition, the mirror descent algorithm with Armijo line search always converges. Numerical results showed that, therefore, the mirror descent algorithm with Armijo line search was the fastest guaranteed-to-converge algorithm for quantum state tomography, empirically on real data-sets.