Asynchronous Distributed Optimization with Heterogeneous Regularizations and Normalizations

TL;DR

This paper introduces an asynchronous distributed optimization framework for multi-agent networks with heterogeneous regularizations and normalizations, ensuring convergence despite communication and computation delays.

Contribution

It proposes a novel asynchronous optimization method that handles heterogeneous regularizations and normalizations, with proven convergence and explicit rate analysis.

Findings

Convergence is achieved under mild conditions.

Explicit convergence rates are derived.

Simulation confirms theoretical results.

Abstract

As multi-agent networks grow in size and scale, they become increasingly difficult to synchronize, though agents must work together even when generating and sharing different information at different times. Targeting such cases, this paper presents an asynchronous optimization framework in which the time between successive communications and computations is unknown and unspecified for each agent. Agents' updates are carried out in blocks, with each agent updating only a small subset of all decision variables. To provide robustness to asynchrony, each agent uses an independently chosen Tikhonov regularization. Convergence is measured with respect to a weighted block-maximum norm in which convergence of agents' blocks can be measured in different p-norms and weighted differently to heterogeneously normalize problems. Asymptotic convergence is shown and convergence rates are derived explicitly in terms of a problem's parameters, with only mild restrictions imposed upon them. Simulation results are provided to verify the theoretical developments made.