Automated Performance Estimation for Decentralized Optimization via Network Size Independent Problems

TL;DR

This paper introduces a new performance estimation framework for decentralized optimization that remains scalable regardless of network size, enabling automatic worst-case analysis of various algorithms.

Contribution

It presents a network size independent PEP formulation that simplifies performance analysis of decentralized methods like DGD, DIGing, and EXTRA.

Findings

Provides tight performance guarantees valid for any network size.

Enables automatic worst-case performance computation via SDP.

Decouples consensus subspace for scalable analysis.

Abstract

We develop a novel formulation of the Performance Estimation Problem (PEP) for decentralized optimization whose size is independent of the number of agents in the network. The PEP approach allows computing automatically the worst-case performance and worst-case instance of first-order optimization methods by solving an SDP. Unlike previous work, the size of our new PEP formulation is independent of the network size. For this purpose, we take a global view of the decentralized problem and we also decouple the consensus subspace and its orthogonal complement. We apply our methodology to different decentralized methods such as DGD, DIGing and EXTRA and obtain numerically tight performance guarantees that are valid for any network size.