Communication-Efficient Distributed Graph Clustering and Sparsification under Duplication Models

TL;DR

This paper develops communication-efficient algorithms for distributed graph clustering and sparsification that handle edge duplication, achieving near-optimal communication costs and quality guarantees in distributed models.

Contribution

It introduces the first communication-optimal algorithms for distributed graph clustering and sparsification under duplication, with tight bounds and quality guarantees.

Findings

Achieves near-matching lower and upper bounds for communication costs.

Provides algorithms with clustering quality close to centralized methods.

First to investigate distributed graph spanner constructions with tight bounds.

Abstract

In this paper, we consider the problem of clustering graph nodes and sparsifying graph edges over distributed graphs, when graph edges with possibly edge duplicates are observed at physically remote sites. Although edge duplicates across different sites appear to be beneficial at the first glance, in fact they could make the clustering and sparsification more complicated since potentially their processing would need extra computations and communications. We propose the first communication-optimal algorithms for two well-established communication models namely the message passing and the blackboard models. Specifically, given a graph on $n$ nodes with edges observed at $s$ sites, our algorithms achieve communication costs $\tilde{O}(ns)$ and $\tilde{O}(n+s)$ ($\tilde{O}$ hides a polylogarithmic factor), which almost match their lower bounds, $\Omega(ns)$ and $\Omega(n+s)$, in the message passing and the blackboard models respectively. The communication costs are asymptotically the same as those under non-duplication models, under an assumption on edge distribution. Our algorithms can also guarantee clustering quality nearly as good as that of centralizing all edges and then applying any standard clustering algorithm. Moreover, we perform the first investigation of distributed constructions of graph spanners in the blackboard model. We provide almost matching communication lower and upper bounds for both multiplicative and additive spanners. For example, the communication lower bounds of constructing a $(2k-1)$-spanner in the blackboard with and without duplication models are $\Omega(s+n^{1+1/k}\log s)$ and $\Omega(s+n^{1+1/k}\max\{1,s^{-1/2-1/(2k)}\log s\})$ respectively, which almost match the upper bound $\tilde{O}(s+n^{1+1/k})$ for both models.