# Generalized Momentum-Based Methods: A Hamiltonian Perspective

## TL;DR

This paper introduces a Hamiltonian framework to unify and analyze a broad class of momentum-based optimization methods, providing new insights into their convergence properties in various settings.

## Contribution

It generalizes classical momentum methods using Hamiltonian dynamics, offering a unified, nonasymptotic convergence analysis for convex and nonconvex optimization.

## Key findings

- Unified Hamiltonian perspective for momentum methods
- Nonasymptotic convergence analysis in convex and nonconvex settings
- Applicable to constrained and non-Euclidean optimization

## Abstract

We take a Hamiltonian-based perspective to generalize Nesterov's accelerated gradient descent and Polyak's heavy ball method to a broad class of momentum methods in the setting of (possibly) constrained minimization in Euclidean and non-Euclidean normed vector spaces. Our perspective leads to a generic and unifying nonasymptotic analysis of convergence of these methods in both the function value (in the setting of convex optimization) and in norm of the gradient (in the setting of unconstrained, possibly nonconvex, optimization). Our approach relies upon a time-varying Hamiltonian that produces generalized momentum methods as its equations of motion. The convergence analysis for these methods is intuitive and is based on the conserved quantities of the time-dependent Hamiltonian.