Inducing Uniform Asymptotic Stability in Non-Autonomous Accelerated Optimization Dynamics via Hybrid Regularization

TL;DR

This paper introduces a hybrid regularization framework for non-autonomous accelerated optimization dynamics, ensuring uniform asymptotic stability and robustness, which are crucial for feedback-based control implementations.

Contribution

The paper proposes a hybrid regularization approach that stabilizes and robustifies continuous-time accelerated optimization dynamics, a novel method for improving their practical reliability.

Findings

Hybrid regularization induces uniform asymptotic stability.

Robustness properties are established for hybrid dynamics.

Discretization mechanisms retain stability and robustness.

Abstract

There have been many recent efforts to study accelerated optimization algorithms from the perspective of dynamical systems. In this paper, we focus on the robustness properties of the time-varying continuous-time version of these dynamics. These properties are critical for the implementation of accelerated algorithms in feedback-based control and optimization architectures. We show that a family of dynamics related to the continuous-time limit of Nesterov's accelerated gradient method can be rendered unstable under arbitrarily small bounded disturbances. Indeed, while solutions of these dynamics may converge to the set of optimizers, in general, this set may not be uniformly asymptotically stable. To induce uniformity, and robustness as a byproduct, we propose a framework where we regularize the dynamics by using resetting mechanisms that are modeled by well-posed hybrid dynamical systems. For these hybrid dynamics, we establish uniform asymptotic stability and robustness properties, as well as convergence rates that are similar to those of the non-hybrid dynamics. We finish by characterizing a family of discretization mechanisms that retain the main stability and robustness properties of the hybrid algorithms.