A Systems-Theoretic View on the Convergence of Algorithms under Disturbances

TL;DR

This paper provides a systems-theoretic framework to analyze the stability and convergence of algorithms operating under disturbances in complex systems, extending classical guarantees to more realistic scenarios.

Contribution

It introduces a systematic approach using converse Lyapunov theorems to derive stability bounds and convergence rates for algorithms affected by disturbances and interconnections.

Findings

Derived stability bounds and convergence rates under disturbances.

Unified framework applicable to distributed learning, privacy, and sensitivity analysis.

Demonstrated the approach's utility across various applications.

Abstract

Algorithms increasingly operate within complex physical, social, and engineering systems where they are exposed to disturbances, noise, and interconnections with other dynamical systems. This article extends known convergence guarantees of an algorithm operating in isolation (i.e., without disturbances) and systematically derives stability bounds and convergence rates in the presence of such disturbances. By leveraging converse Lyapunov theorems, we derive key inequalities that quantify the impact of disturbances. We further demonstrate how our result can be utilized to assess the effects of disturbances on algorithmic performance in a wide variety of applications, including communication constraints in distributed learning, sensitivity in machine learning generalization, and intentional noise injection for privacy. This underpins the role of our result as a unifying tool for algorithm analysis in the presence of noise, disturbances, and interconnections with other dynamical systems.