Semi-MapReduce Meets Congested Clique

TL;DR

This paper introduces semi-MapReduce (semiMPC), a model for graph algorithms that is shown to be equivalent to the congested clique model, enabling improved distributed algorithms and efficient simulations across models.

Contribution

The paper demonstrates the equivalence of semiMPC and the congested clique model and shows how this facilitates improved algorithms and simulations across distributed computing models.

Findings

semiMPC is equivalent to the congested clique model

Algorithms in CONGEST can be simulated efficiently in semiMPC

Improved algorithms are derived for distributed models using this equivalence

Abstract

Graph problems are troublesome when it comes to MapReduce. Typically, to be able to design algorithms that make use of the advantages of MapReduce, assumptions beyond what the model imposes, such as the density of the input graph, are required. In a recent shift, a simple and robust model of MapReduce for graph problems, where the space per machine is set to be O(|V|), has attracted considerable attention. We term this model semi-MapReduce, or in short, semiMPC, and focus on its computational power. We show through a set of simulation methods that semiMPC is, perhaps surprisingly, equivalent to the congested clique model of distributed computing. However, semiMPC, in addition to round complexity, incorporates another practically important dimension to optimize: the number of machines. Furthermore, we show that algorithms in other distributed computing models, such as CONGEST, can be simulated to run in the same number of rounds of semiMPC while also using an optimal number of machines. We later show the implications of these simulation methods by obtaining improved algorithms for these models using the recent algorithms that have been developed.