Resource-Aware Discretization of Accelerated Optimization Flows

TL;DR

This paper introduces resource-aware discretization methods for accelerated optimization flows that preserve convergence properties by leveraging control-inspired techniques like Lyapunov functions and variable stepsizes.

Contribution

It proposes a novel discretization approach based on resource-aware control principles, ensuring stability and performance in digital implementations of optimization algorithms.

Findings

Discrete algorithms retain Lyapunov decrease properties.

Variable stepsize schemes adapt to system dynamics.

Approach applicable to various stable flows with Lyapunov certificates.

Abstract

This paper tackles the problem of discretizing accelerated optimization flows while retaining their convergence properties. Inspired by the success of resource-aware control in developing efficient closed-loop feedback implementations on digital systems, we view the last sampled state of the system as the resource to be aware of. The resulting variable-stepsize discrete-time algorithms retain by design the desired decrease of the Lyapunov certificate of their continuous-time counterparts. Our algorithm design employs various concepts and techniques from resource-aware control that, in the present context, have interesting parallelisms with the discrete-time implementation of optimization algorithms. These include derivative- and performance-based triggers to monitor the evolution of the Lyapunov function as a way of determining the algorithm stepsize, exploiting sampled information to enhance algorithm performance, and employing high-order holds using more accurate integrators of the original dynamics. Throughout the paper, we illustrate our approach on a newly introduced continuous-time dynamics termed heavy-ball dynamics with displaced gradient, but the ideas proposed here have broad applicability to other globally asymptotically stable flows endowed with a Lyapunov certificate.