Analysis Accelerated Mirror Descent via High-resolution ODEs

TL;DR

This paper introduces high-resolution ODEs for accelerated mirror descent, providing a detailed analysis of its convergence properties and demonstrating its efficiency in minimizing the squared gradient norm at an inverse cubic rate.

Contribution

The paper develops high-resolution ODEs for accelerated mirror descent and a Lyapunov framework to analyze its convergence, distinguishing it from low-resolution models.

Findings

High-resolution ODEs differentiate between heavy-ball and Nesterov methods.

Accelerated mirror descent achieves inverse cubic rate in squared gradient norm.

Lyapunov framework effectively analyzes convergence in continuous and discrete time.

Abstract

Mirror descent plays a crucial role in constrained optimization and acceleration schemes, along with its corresponding low-resolution ordinary differential equations (ODEs) framework have been proposed. However, the low-resolution ODEs are unable to distinguish between Polyak's heavy-ball method and Nesterov's accelerated gradient method. This problem also arises with accelerated mirror descent. To address this issue, we derive high-resolution ODEs for accelerated mirror descent and propose a general Lyapunov function framework to analyze its convergence rate in both continuous and discrete time. Furthermore, we demonstrate that accelerated mirror descent can minimize the squared gradient norm at an inverse cubic rate.