How Clustering Affects the Convergence of Decentralized Optimization over Networks: A Monte-Carlo-based Approach

TL;DR

This paper investigates how the clustering coefficient in network topologies influences the convergence rate of decentralized optimization algorithms, revealing that lower clustering leads to faster convergence, which can enhance machine learning performance.

Contribution

The study introduces a Monte-Carlo simulation approach to analyze the impact of clustering on convergence, comparing synthetic and real-world networks while controlling other network properties.

Findings

Lower clustering coefficient correlates with higher convergence rate.

Monte-Carlo simulations validate the impact across various network topologies.

Real-world network analysis supports the simulation results.

Abstract

Decentralized algorithms have gained substantial interest owing to advancements in cloud computing, Internet of Things (IoT), intelligent transportation networks, and parallel processing over sensor networks. The convergence of such algorithms is directly related to specific properties of the underlying network topology. Specifically, the clustering coefficient is known to affect, for example, the controllability/observability and the epidemic growth over networks. In this work, we study the effects of the clustering coefficient on the convergence rate of networked optimization approaches. In this regard, we model the structure of large-scale distributed systems by random scale-free (SF) and clustered scale-free (CSF) networks and compare the convergence rate by tuning the network clustering coefficient. This is done by keeping other relevant network properties (such as power-law degree distribution, number of links, and average degree) unchanged. Monte-Carlo-based simulations are used to compare the convergence rate over many trials of SF graph topologies. Furthermore, to study the convergence rate over real case studies, we compare the clustering coefficient of some real-world networks with the eigenspectrum of the underlying network (as a measure of convergence rate). The results interestingly show higher convergence rate over low-clustered networks. This is significant as one can improve the learning rate of many existing decentralized machine-learning scenarios by tuning the network clustering.