Efficient Distributed Workload (Re-)Embedding

TL;DR

This paper introduces a distributed online algorithm for dynamic workload embedding in reconfigurable networked systems, balancing the benefits of collocation against re-embedding costs, and nearly matches the theoretical optimal performance.

Contribution

It presents a novel distributed online algorithm for workload embedding that is asymptotically almost optimal, addressing the challenge of dynamic, demand-aware reconfiguration.

Findings

Algorithm is asymptotically almost optimal in competitive ratio.

The approach effectively balances reconfiguration costs and performance benefits.

Application demonstrated in distributed union find problem.

Abstract

Modern networked systems are increasingly reconfigurable, enabling demand-aware infrastructures whose resources can be adjusted according to the workload they currently serve. Such dynamic adjustments can be exploited to improve network utilization and hence performance, by moving frequently interacting communication partners closer, e.g., collocating them in the same server or datacenter. However, dynamically changing the embedding of workloads is algorithmically challenging: communication patterns are often not known ahead of time, but must be learned. During the learning process, overheads related to unnecessary moves (i.e., re-embeddings) should be minimized. This paper studies a fundamental model which captures the tradeoff between the benefits and costs of dynamically collocating communication partners on $\ell$ servers, in an online manner. Our main contribution is a distributed online algorithm which is asymptotically almost optimal, i.e., almost matches the lower bound (also derived in this paper) on the competitive ratio of any (distributed or centralized) online algorithm. As an application, we show that our algorithm can be used to solve a distributed union find problem in which the sets are stored across multiple servers.

Equations

$$\Delta =\min \{|V_{\mathrm{init}}(S_0)\cap V_0|,|V_{\mathrm{init}}(S_0)\cap V_1|\}.$$

$$\Delta =\underset{i=0,1}{\max }\underset{j=0,1}{\min }|V_{\mathrm{init}}(S_i)\cap V_j|.$$

$$\sum _{C\in X}O(\alpha |C|\log |C|)\le \sum _{C\in X}O(\alpha |C|\log \Delta )=O(\alpha \Delta \log \Delta ).$$

$$\frac{\mathrm{ON}}{\mathrm{OPT}}=\frac{O(\alpha \Delta \log \Delta )}{2\alpha \Delta }=O(\log \Delta )=O(\log n).$$

$$\frac{\mathrm{ON}}{\mathrm{OPT}}$$

$$\mathrm{APPROX}=\sum _i{\mathrm{APPROX}}_i\le 2\sum _i{\mathrm{OPT}}_i=2\cdot \mathrm{OPT}.$$

$$\frac{{\mathrm{APPROX}}_i}{{\mathrm{OPT}}_i}\le \frac{\alpha (W+n_{i0}+n_{i1})}{\alpha (W+n_{i0})}\le \frac{W+2n_{i0}}{W+n_{i0}}\le 2.$$

$$\overline{\#w_j(C)}$$

$$\ge |C_1\cup C_2|-|C_1\cup C_2|/2=|C_1\cup C_2|/2\ge |C|/4.$$

$$|C_v|/4\ge 2|C_v|/8\ge |C_u\cup C_v|/8=|C|/8.$$

$$\sum _{i}O({\mathrm{APPROX}}_i\cdot \log n)=O(\mathrm{APPROX}\cdot \log n)=O(\mathrm{OPT}\cdot \log n).$$

$$\sum _{C\in X}O(\alpha |C|\log |C|)\le O(\alpha \Delta \log n)=O({\mathrm{ON}}_i\cdot \log n).$$

$$\sum _{C\in Y}O(\alpha |C|\log |C|)\le \sum _{C\in Y}O(\alpha |C|\log n)=O(\alpha n\log n).$$

$$\mathrm{OPT}\ge \sum _{C\in X}\Omega (\alpha \#w_1(C))=\Omega (\alpha \epsilon n/(\ell \log \ell )).$$

$$\frac{O(\alpha n\log n+(\mathrm{OPT}\cdot \ell \log n)/\epsilon )}{\mathrm{OPT}}=\frac{O(\alpha n\log n)}{\mathrm{OPT}}+O\left(\frac{\ell \log n}{\epsilon }\right)$$

$$\le O\left(\frac{\alpha n\log n\cdot \ell \log \ell }{\alpha \epsilon n}\right)+O\left(\frac{\ell \log n}{\epsilon }\right)=O\left(\frac{\ell \log n\log \ell }{\epsilon }\right).\text{qed}$$

$$O(({\ell }^2\log n)/\epsilon )+n\log n+(M\ell \log n)/\epsilon )$$

$$=O(n\log n+(M\ell \log n)/\epsilon ),$$

$$\frac{O(\alpha n\log n+(\alpha n\ell \log n)/(\epsilon -{\epsilon }^{\prime }))}{\mathrm{OPT}}=O\left(\frac{\alpha n\log n+(\alpha n\ell \log n)/(\epsilon -{\epsilon }^{\prime })}{\alpha \epsilon n/(\ell \log \ell )}\right)=O\left(\frac{{\ell }^2\log n\log \ell }{{\epsilon }^2}\right),$$

$$y_i=(1+\epsilon )n/2^{i+1}\le 1.98\cdot 2^{a-i-1}$$

$$z=2^{a-i}-2p\le y_i.$$

$$p$$

$$=2^{a-i-1}(1-0.99)$$

$$=0.01\cdot 2^{a-i-1}.$$

$$\Omega (\alpha \cdot 2^{a-i}\cdot 2^i)=\Omega (\alpha \cdot 2^{a-2})=\Omega (\alpha n).$$

Full text

1 Introduction

Along with the trend towards more data centric applications (e.g., online services like web search, social networking, financial services as well as emerging applications such as distributed machine learning [41, 37]), comes a need to scale out such applications, and distribute the workload across multiple servers or even datacenters. However, while such parallel processing can improve performance, it can entail a non-trivial load on the interconnecting network. Indeed, distributed cloud applications, such as batch processing, streaming, or scale-out databases, can generate a significant amount of network traffic [40].

At the same time, emerging networked systems are becoming increasingly flexible and thereby provide novel opportunities to mitigate the overhead that distributed applications impose on the network. In particular, the more flexible and dynamic resource allocation (enabled, e.g., by virtualization) introduces a vision of workload-aware infrastructures which optimize themselves to the demand [9]. In such infrastructures, communication partners which interact intensively, may be moved closer (e.g., collocated on the same server, rack, or datacenter) in an adaptive manner, depending on the demand. This “re-embedding” of the workload allows to keep communication local and reduce costs. Indeed, empirical studies have shown that communication patterns in distributed applications feature much locality, which highlights the potential of such self-adjusting networked systems [29, 47, 11].

However, leveraging such resource reconfiguration flexibilities to optimize performance, poses an algorithmic challenge. First, while collocating communication partners reduces communication cost, it also introduces a reconfiguration cost (e.g., due to virtual machine migration). Thus, an algorithm needs to strike a balance between the benefits and the cost of such reconfigurations. Second, as workloads and communication patterns are usually not known ahead of time, reconfiguration decisions need to be made in an online manner, i.e., without knowing the future. We are hence in the realm of online algorithms and competitive analysis.

This paper studies the fundamental tradeoff underlying the optimization of such workload-aware reconfigurable systems. In particular, we consider the design of an online algorithm which, without prior knowledge of the workload, aims to minimize communication cost by performing a small number of moves (i.e., migrations). In a nutshell (more details will follow below), we consider a communication graph between $n$ vertices (e.g., virtual machines) which can be perfectly partitioned among a set of $\ell$ servers (resp. racks or datacenters) of a given capacity. We assume that the communication patterns, which partition the communication graph, consist of $n/\ell$ vertices and that once the whole communication graph was revealed, each server must contain exactly one communication pattern.

The communication graph is initially unknown and revealed to the algorithm in an online manner, edge-by-edge, by an adversary who aims to maximize the cost of the given algorithm. The cost here consists of communication cost and moving cost: The algorithm incurs one unit cost if the two endpoints (i.e., communication partners) of the request belong to different servers. After each request, the algorithm can reconfigure the infrastructure and move communication endpoints from one server to another, essentially repartitioning the communication partners; however, each move incurs a cost of $\alpha>1$.

In other words, this paper considers the problem of learning a partition, i.e., an optimal assignment of communication partners to servers, at low communication and moving cost. Interestingly, while the problem is natural and fundamental, not much is known today about the algorithmic challenges underlying this problem, except for the negative result that no good competitive algorithm can exist if communication partners can change arbitrarily over time [8]. This lower bound motivates us, in this paper, to focus on the online learning variant where the communication partners are unknown but fixed. At the same time, as we will show, the problem features interesting connections to several classic problems. Specifically, the problem can be seen as a distributed version of classic online caching problems [51] or an online version of the $k$-way partitioning problem [48].

1.1 Our Contributions

We initiate the study of a fundamental problem, how to learn and re-embed workload in an online manner, with few moves. We make the following main contributions.

We present a distributed $O((\ell\log\ell\log n)/\varepsilon)$-competitive online algorithm for servers of capacity $(1+\varepsilon)n/\ell$, where $\varepsilon\in(0,1/2)$. We allow the servers to have $\varepsilon n/\ell$ more space than is strictly needed to embed its corresponding communication pattern (which is of size $n/\ell$); we denote this additional space as augmentation. Such augmentation is also needed, as our lower bounds discussed next show.

We show that there are inherent limitations of what online algorithms can achieve in our model: We derive a lower bound of $\Omega(1/\varepsilon+\log n)$ on the competitive ratio of any deterministic online algorithm given servers of capacity at least $(1+\varepsilon)n/\ell$. This lower bound has several consequences: (1) To obtain $O(\log n)$-competitive algorithms, the servers must have $\Omega(n/(\ell\log n))$ augmentation. (2) If the servers have $\Omega(n/\ell)$ augmentation (e.g., each server has 10% more capacity than the size of its communication pattern), our algorithm is optimal up to an $O(\ell\log\ell)$ factor. Thus, our results are particularly interesting for large servers, e.g., in a wide-area networking context where there is usually only a small number of datacenters where communication partners can be collocated (e.g., $\ell=20$): if each datacenter (“server”) has augmentation $0.1\cdot n/\ell$ , our algorithm is optimal up to constant factors.

The distributed algorithms we present not only provide good competitive ratios but they are also highly efficient w.r.t. the network traffic they cause. In fact, we show that for $\ell=O(\sqrt{\varepsilon n})$ servers, running the algorithms introduces only little overhead in network traffic and that this overhead is asymptotically negligible (see Section 5.1).

While the previous algorithms require exponential time, we also present polynomial time algorithms at the cost of a slightly worse competitive ratio of $O((\ell^{2}\log n\log\ell)/\varepsilon^{2})$ in Section 5.2.

As a sample application of our newly introduced model we present a distributed union find data structure [27, 52] (also known as disjoint-set data structure or merge-find data structure) in Section 7.1: There are $n$ items from a universe which are distributed over $\ell$ servers; each server can store at most $(1+\varepsilon)n/\ell$ items and each item belongs to a unique set. The operation union allows to merge two sets. In our setting, we require that items from the same set must be assigned to the same server. To reduce the network traffic, our goal is to minimize the number of item moves during union operations. For example, when two sets are merged which are assigned to different servers, then the items of one of the sets must be reassigned to another server. We compare against an optimal offline algorithm which knows the initial assignment of all items and all union operations in advance. We obtain the same competitive ratios as above. We believe that this distributed union find data structure will be useful as a subroutine for several problems such as merging duplicate websites in search engines [17].

We also show that our algorithms solve an online version of the $k$-way partition problem in Section 7.2.

1.2 Organization

We introduce our model formally in Section 2. To ease the readability, we first explore centralized online algorithms that efficiently collocate communication patterns for $\ell=2$ servers in Section 3, and then study the general case of $\ell>2$ servers in Section 4. In Section 5 we show how the previously derived centralized algorithms can be made distributed and how the algorithm can be implemented in polynomial time at the cost of a slightly worse competitive ratio. We provide the lower bounds in Section 6. Section 7 provides a distributed union find data structure and a result for online $k$-way partitioning; these problems serve as sample applications of the problem we study. After reviewing related work in Section 8, we conclude our contribution in Section 9.

2 Model

We start by formally introducing the model which we will be studying in this paper. We consider a set of vertices $V$ (e.g., a set of virtual machines) which interact according to an initially unknown communication pattern, which can be represented as a communication graph $G=(V,E)$ with $n=|V|$ vertices and $m=|E|$ edges. The vertices of $G$ are partitioned into $\ell$ sets $V_{0},\dots,V_{\ell-1}$ where each $V_{i}$, forming a connected communication component (the workload), has size222Note that in general $n/\ell$ is not always an integer and we would have to take rounding into account. However, we ignore this technicality for better readability of the paper. $n/\ell$; the connected components of $G$ coincide with the sets $V_{i}$. The sets $V_{i}$ are the communication patterns which need to be recovered by the online algorithm, henceforth called ground truth components.

The communicating vertices $V$ need to be assigned to $\ell$ servers $S_{0},\dots,S_{\ell-1}$. Accordingly, we define an assignment (the embedding) which is a function from the vertices to the servers. The load of a server $S_{j}$ is the number of vertices that are assigned to it. An assignment is valid if each server has load at most $n/\ell+K$ and we call $n/\ell+K$ the capacity of the servers and $K$ the augmentation. If $K=0$, the total server capacity exactly matches the number of vertices. The available capacity of a server is the difference between the server’s capacity and its load. An assignment is perfectly balanced if each server has load exactly $n/\ell$. We assume that when the algorithm starts, we have a perfectly balanced assignment. We will write $V(S_{j})$ to denote the set of vertices assigned to server $S_{j}$ and $V_{\operatorname{init}}(S_{j})$ for the set of vertices initially assigned to server $S_{j}$. We say that an assignment is a perfect partitioning if it satisfies $\{V(S_{0}),\dots,V(S_{\ell-1})\}=\{V_{0},\dots,V_{\ell-1}\}$, i.e., the vertices on the servers coincide with the connected components of $G$.

The communication graph $G=(V,E)$ is revealed by an adversary in an online manner, as a sequence of edges $\sigma=(e_{1},\dots,e_{r})$, where $r$ denotes the number of communication requests and $e_{i}\in E$ for each $i$. Note that the adversary can only provide edges which are present in $E$ and that each edge can appear multiple times in the sequence of edges. We assume that the sequence of the edges provided by the adversary reveals the ground truth components $V_{i}$, i.e., after having seen all edges in $\sigma$ the algorithm can compute the connected components of $G$ which (by assumption) coincide with the ground truth components $V_{i}$. We present an illustration of the model in Figure 1.

Now an online algorithm must iteratively change the assignment such that eventually the assignment is a perfect partitioning.

The reassignment needs to be done while minimizing certain communication and migration cost. If an edge $e=(u,v)$ provided by the adversary has both endpoints in the same server $S_{i}$ at the time of the request, an algorithm incurs no costs. If $u$ and $v$ are in different servers $S_{i}$ and $S_{j}$, then their communication cost is $1$. Reassigning, i.e., moving, a vertex $u$ from a server $S_{i}$ to a server $S_{j}$ costs $\alpha>1$.

When measuring the cost of an online algorithm, we will compare against an optimal offline algorithm denoted by OPT . OPT has a priori knowledge of the communication graph $G=(V,E)$ as well as the given the sequence of all edges $\sigma=(e_{1},\dots,e_{r})$. In other words, OPT can compute the assignment of vertices to servers which provides the minimum migration cost from the initial assignment.

Now let the cost paid by an online algorithm be denoted by ON and let the cost of the optimal offline algorithm be denoted by OPT . We consider the design of an online algorithm $\operatorname{ON}$ which minimizes the (strict) competitive ratio defined as $\frac{\operatorname{ON}}{\operatorname{OPT}}$.

The Role of Connected Components

We will briefly discuss how connected components are induced by subsequence of $\sigma$ and how we will treat connected components in our algorithms. We then give a reduction which helps us to avoid considering communication costs in our proofs.

Recall that the adversary provides a sequence of edges $\sigma$ to an algorithm in an online manner. As this happens, an algorithm can keep track of all edges it has seen so far. Let this set of edges be $E^{\prime}$. Using the edges in $E^{\prime}$, the algorithm can compute the connected components $C_{1},\dots,C_{q}$ which are induced by $E^{\prime}$. Here, $q$ denotes the current number of connected components.

To obtain a better understanding of the relationship between the connected components $C_{i}$ and the ground truth components $V_{j}$, we make four observations: (1) When the algorithm starts, all connected components $C_{i}=\{v_{i}\}$ only consist of single vertices (because $\sigma$ has not yet revealed any edges). (2) When a previously unknown edge $e=(u,v)$ is revealed which has its endpoints in different connected components $C_{u}$ and $C_{v}$, these connected components get merged. (3) Suppose a subsequence of $\sigma$ induces $q>\ell$ connected components $C_{i}$ (i.e., $\sigma$ has not yet revealed the whole graph $G$). Then for each ground truth component $V_{j}$ there exists a subset $\mathbb{C}\subset\{C_{1},\dots,C_{q}\}$ of the connected components such that $V_{j}=\bigcup_{C\in\mathbb{C}}C$. (4) When an algorithm terminates (and, hence, $\sigma$ revealed all edges in $E$), there exists a one-to-one correspondence between the connected components $C_{i}$ and the ground truth components $V_{j}$.

By assumption on the input from the adversary, when all of $\sigma$ was revealed, $E^{\prime}$ reveals the ground truth components $V_{0},\dots,V_{\ell-1}$. Thus, in total there will be exactly $n-\ell$ edges connecting vertices from different connected components.

All of the algorithms we consider in this paper have the property that they always assign vertices of the same connected component to the same server. This property implies that the communication cost paid by such an algorithm is bounded by its moving cost (we prove this in the following lemma). Hence, in the rest of the paper we only need to bound the moving costs of our algorithms to obtain a bound on their total costs.

Lemma 1**.**

Suppose an algorithm $\mathcal{A}$ always assigns all vertices of the same connected component to the same server and pays $\mathscr{C}$ for moving vertices. Then its communication cost is at most $\mathscr{C}$. Furthermore, its total cost is at most $2\mathscr{C}$.

Proof.

Suppose the adversary provides an edge $(u,v)$. We consider two cases. Case 1: $u$ and $v$ are assigned to the same server. Then $\mathcal{A}$ does not pay any communication costs. Case 2: $u$ and $v$ are assigned to connected components $C_{u}$ and $C_{v}$ on different servers. Then the algorithm needs to pay $1$ communication cost. However, in this case $\mathcal{A}$ must move $C_{u}$ or $C_{v}$ to a different server at the cost of at least $\alpha>1$. Hence, the moving cost is larger than the communication cost. We conclude that $\mathcal{A}$’s total communication cost is at most $\mathscr{C}$. By summing the two quantities, we obtain the second claim of the lemma. ∎

While in Lemma 1 we have shown that algorithms which always collocate connected components immediately are efficient w.r.t. their total cost, in Section 6.1 we show that any efficient algorithm must satisfy a similar (slightly more general) property.

Throughout the rest of the paper, we write $|C|$ to denote the number of vertices in a connected component $C$. For a vertex $u$, we write $C_{u}$ to denote the connected component $C$ which contains $u$.

3 Online Partition for Two Servers

In this section, we consider the problem of learning a communication graph with few moves with two servers. As we will see later, the concepts introduced in this section will be useful when solving the problem with $\ell>2$ servers. We derive the following result.

Theorem 2**.**

Consider the setting with two servers of capacity $(1+\varepsilon)n/2$ for $\varepsilon\in(0,1)$, i.e., the augmentation is $\varepsilon n/2$. Then there exists an algorithm with competitive ratio $O((\log n)/\varepsilon)$.

The proof is organized as follows. We first characterize the optimal solution by OPT in Section 3.1. We then present an algorithm which is efficient whenever OPT incurs “significant cost”, in Section 3.2. In Section 3.3, we describe an algorithm which is efficient whenever the solution by OPT is “cheap”. We prove Theorem 2 via a combination of the two algorithms in Section 3.4.

3.1 Costs of OPT

The following lemma gives a precise characterization of the cost paid by OPT in the two server case. It introduces a parameter $\Delta$ which equals the number of vertices moved by OPT and which we will be using throughout the rest of this section.

Lemma 3**.**

Suppose $\ell=2$ and the vertices initially assigned to the servers $S_{i}$ are given by the sets $V_{\operatorname{init}}(S_{i})$ for $i=0,1$. Then the cost of OPT is $2\alpha\Delta$, where

[TABLE]

It follows immediately that $\Delta\leq n/4$ (as $|V_{\operatorname{init}}(S_{0})|=n/2$).

Proof.

Recall that our model forces OPT to provide a final assignment satisfying $\{V(S_{0}),V(S_{1})\}=\{V_{0},V_{1}\}$, i.e., OPT must produce a final assignment which coincides with the ground truth components (even if paying for each communication request individually and not relocating any vertices might be cheaper). Thus, we can assume that OPT performs all vertex moves in the beginning, to avoid paying any communication cost. Since the edge sequence $\sigma=(e_{1},\dots,e_{r})$ provided by the adversary is assumed to reveal the connected components $V_{0}$ and $V_{1}$, OPT can compute $V_{0}$ and $V_{1}$ before it performs any moves.

As there are only two servers, one of them must contain at least half of the vertices from $V_{0}$ in the initial assignment. Now let us first assume that this server is $S_{0}$; this setting is illustrated in Figure 2. In this case, OPT can move the $\Delta$ vertices in $V_{\operatorname{init}}(S_{0})\cap V_{1}$ to $S_{1}$ and those in $V_{\operatorname{init}}(S_{1})\cap V_{0}$ to $S_{0}$. It is easy to verify that this yields an assignment satisfying $\{V(S_{0}),V(S_{1})\}=\{V_{0},V_{1}\}$ and that the moving cost is minimized. Further, the cost for this reassignment is exactly $2\alpha\Delta$.

The second case where $S_{1}$ contains more than half of the vertices from $V_{0}$ in the initial assignment is symmetric. ∎

While in Lemma 3 we have presented the lower bound w.r.t. server $S_{0}$, we could also express the lower bound in terms of server $S_{1}$. We then obtain the following equality:

[TABLE]

3.2 The Small–Large–Rebalance Algorithm

A natural idea to obtain a small number of vertex moves is to proceed as follows. Whenever two vertices $u$ and $v$ belonging to different connected components communicate, the algorithm merges their connected components. If the two components were already assigned to the same server, no vertex moves are required. If $u$ and $v$ are assigned to different servers, we move the smaller connected component to the server of the larger connected component. This algorithm is efficient in that it never performs more than $O(n\log n)$ vertex moves (see Lemma 4).

However, the algorithm could require much augmentation, as it does not account for server capacities. Thus, we propose the following extension called the Small–Large–Rebalance Algorithm: Whenever a server exceeds its capacity, the algorithm computes a perfectly balanced assignment of the vertices which respects the previously observed connected components; we call this a rebalancing step. We provide pseudocode in Algorithm 1.

Section 5.2.1 shows how such a rebalancing step can be implemented in $O(n^{2})$ time. Later, we show that there can be at most $O((\log n)/\varepsilon)$ such rebalancing steps which implies that the total running time Algorithm 1 is $O((n^{2}\log n)/\varepsilon)$.

Note that Algorithm 1 also works in the setting with $\ell$ servers for $\ell>2$. We will analyze this more general algorithm in Section 4.4.

3.2.1 Analysis

To analyze Algorithm 1, we first consider the algorithm from the first paragraph which does not have the rebalancing step. When the algorithm moves a smaller component to the server of a larger component, we call this a small-to-large step.

Lemma 4**.**

Consider the algorithm which always moves the smaller connected component to the server of the larger connected component when it obtains an edge between vertices from different connected components. The algorithm moves each vertex at most $O(\log n)$ times. Its total number of vertex moves is $O(n\log n)$.

Proof.

Consider any vertex $v$. We use the following accounting: Whenever $v$ is in the a smaller component that is moved, add a token to $v$. Now observe that whenever $v$ gains a token, the size of its component at least doubles. This implies that $v$ can be in the smaller component only $O(\log n)$ times. Thus, $v$ cannot accumulate more than $O(\log n)$ tokens. Since this holds for each of the $n$ vertices, the total number of moves is $O(n\log n)$. ∎

The following lemma provides the analysis for Algorithm 1 which performs small-to-large steps and rebalancing steps.

Lemma 5**.**

Suppose both servers have capacity $(1+\varepsilon)n/2$, i.e., the augmentation is $\varepsilon n/2$ for $\varepsilon\in(0,1)$. Then Algorithm 1 performs $O((\log n)/\varepsilon)$ rebalancing steps and $O((n\log n)/\varepsilon)$ vertex moves.

Proof.

We prove the bound on the number of vertex moves; the claim about the number of rebalancing steps is proved along the way. Note that all vertex moves performed by the algorithm originate from either small-to-large steps or from rebalancing steps. We bound the number of each of these vertex moves separately.

Note that the token-based argument from Lemma 4 still applies to the small-to-large steps of Algorithm 1. This implies that the total number of vertex moves due small-to-large steps is $O(n\log n)$.

Now consider the vertex moves caused by the rebalancing steps and recall that the initial assignment is perfectly balanced. Whenever a server exceeds its load, the small-to-large steps of the algorithm must have moved at least $\varepsilon n/2$ vertices (because the augmentation of one of the servers is exceeded). This can only happen $O((n\log n)/(\varepsilon n))=O((\log n)/\varepsilon)$ times since the total number of vertex moves due to small-to-large steps is $O(n\log n)$. Hence, the number of rebalancing steps is at most $O((\log n)/\varepsilon)$. Since each rebalancing step performs $O(n)$ vertex moves, the lemma follows. ∎

3.2.2 More Efficient Rebalancing

We next propose a better rebalancing strategy which makes Algorithm 1 more efficient. So far, we used $\Theta(n)$ vertex moves for each rebalancing operation at the cost of $\Theta(\alpha n)$. We now bring the rebalancing cost down to $O(\operatorname{OPT})$.

We adjust Algorithm 1 in the following way: Instead of rebalancing by taking any perfectly balanced assignment respecting the connected components (Line 12), we choose a perfectly balanced assignment respecting the connected components which minimizes the number of vertex moves from the initial solution. We call such an assignment cheap.

To find a cheap assignment, the algorithm could simply do the following: (1) Recall the initial assignment. (2) Exhaustively enumerate all perfectly balanced assignments respecting the connected components. (3) Among all of these assignments find one which is cheap. While such a simple algorithm can in principle be computationally costly, we can here exploit the online model of computation which allows us unlimited computational power. In Section 5.2 we show how less efficient rebalancing strategies can be implemented in polynomial time and we obtain slightly worse competitive ratios.

With the improved rebalancing strategy, we obtain Proposition 6.

Proposition 6**.**

Suppose all servers have capacity $(1+\varepsilon)n/2$, $\varepsilon>0$. Then the number of vertex reassignments performed by Algorithm 1 with more efficient rebalancing is $O(n\log n+(\Delta\log n)/\varepsilon)$, where $\Delta$ is the number of vertex moves used by OPT .

Proof.

First, note that the number of vertex moves for moving smaller components to larger components (Line 9) is $O(n\log n)$, by exactly the same arguments used in the proof of Lemma 5.

Second, we bound the number of vertex moves required for the rebalancing operations. Whenever the algorithm needs to rebalance, we can assume (for the sake of the analysis) that the algorithm makes the following three steps: (1) Roll back all changes done by small-to-large moves (Line 9) since the last rebalancing operation. Thus, after rolling back we have the same assignment as after the last rebalancing operation. (2) Roll back to the initial assignment (by undoing the last rebalancing operation). (3) Move to a cheap assignment.

Observe that Step (1) and (2) of the previous three step procedure increase the number of vertex moves only by a constant factor compared to when the algorithm does not roll back: In total, Step (1) only adds additional $O(n\log n)$ vertex moves because each small-to-large move is undone exactly once. Step (2) only doubles the number of vertex moves for moving to cheap assignments as each rebalancing is only undone once.

Thus, we can complete the proof if we can show that the total number of vertex moves for moving from the initial assignment to the cheap assignments is bounded by $O((\Delta\log n)/\varepsilon)$.

By Lemma 5, the number of rebalancing steps is bounded by $O((\log n)/\varepsilon)$. Now we argue that for moving from the initial solution to each cheap assignment, the rebalancing moves at most $O(\Delta)$ vertices: Every time the algorithm computes a cheap rebalancing, the final solution obtained by OPT is a perfectly balanced assignment respecting the connected components. Thus, the number of vertex moves to obtain a cheap rebalancing is bounded by the number of moves performed by OPT which is $O(\Delta)$. This finishes the proof. ∎

3.3 The Majority Voting Algorithm

We now present an algorithm which works well whenever the cost paid by OPT is small, i.e., when OPT only needs to move few vertices. The issue with Algorithm 1 from Section 3.2 is that during its execution, it might deviate much from the initial assignment (and thus move many vertices). The following algorithm has the property that it always stays close to the initial assignment.

For ease of readability, we will often refer to the two servers as the left and right servers, respectively, instead of calling them $S_{0}$ and $S_{1}$.

Our algorithm starts by coloring vertices on the left server yellow and on the right server black. Throughout the execution of the algorithm, the vertices will keep this initially assigned color. The algorithm then follows the idea of always moving the smaller connected component to the server of the larger connected component; we will refer to this as small-to-large step. To stay close to the initial assignment, whenever the number of vertices in a newly merged connected component surpasses a power of $2$, the algorithm performs a majority vote and moves the component to the server where more of its vertices originate from. More formally, we say that a set of vertices (e.g., a connected component) has a yellow (black) majority if it contains more yellow (black) vertices than black (yellow) vertices. In the majority voting step, the algorithm moves a component with a yellow (black) majority which is currently on the right (left) server to the left (right) server. The pseudocode for this procedure is stated in Algorithm 2.333Note that in Algorithm 2 the following is possible when a component $C_{u}$ is merged with a component $C_{v}$: $C_{u}$ is moved from $S$ to $S_{v}$ due to a small-to-large step and immediately after that $C_{u}\cup C_{v}$ is moved back to $S$ due to a majority-voting step. Thus, it would be more efficient to compute the result of the majority-voting step earlier and to move $C_{v}$ to $S$ immediately (without ever moving $C_{u}$ to $S_{v}$). This modification would be slightly more efficient but it would affect the competitive ratio of the algorithm only by at most a constant factor. Thus, to simplify our analysis, we ignore this modification.

The reason for introducing the majority voting step is that it keeps the assignments produced by the algorithm during its runtime close to the initial assignment. Due to this property, we can show that the cost of Algorithm 2 is always close to the cost of OPT . The formal guarantees are stated in Proposition 7.

Proposition 7**.**

Let $\Delta$ be the number of vertex moves performed by OPT (see Section 3.1). Then Algorithm 2 is $O(\log n)$-competitive and the load of both servers is bounded by $n/2+4\Delta$.

We devote rest of this subsection to the proof of the proposition. We start bounding the augmentation. For the proofs recall that $V_{0}$ and $V_{1}$ are the ground truth connected components of $G$.

In the following we are interested in what happened to a connected component since its last majority vote. To this end, we decompose it into a sequence of smaller connected components such that first a majority vote is performed and after that, only small-to-large steps are performed. For all of these small-to-large steps, the component will stay on the server that was picked by the majority vote. The following definition makes this notion formal.

Definition 8** (Doubling Decomposition).**

Let $C$ be a connected component and let $s\in\mathbb{N}$ be such that $2^{t}\leq|C|<2^{t+1}$. Consider $k$ disjoint sets of vertices $C_{i}\subseteq V$ and let $\mathbb{C}_{j}=\bigcup_{i=1}^{j}C_{i}$ for $j=1,\dots,k$.

A sequence $(C_{1},\dots,C_{k})$ is a doubling decomposition of $C$ if the following properties hold:

$C=\mathbb{C}_{k}=\bigcup_{i=1}^{k}C_{i}$, 2. 2.

during the execution of the algorithm, first $\mathbb{C}_{1}\cup C_{2}$ are merged, then $\mathbb{C}_{2}\cup C_{3}$ are merged, and, more generally, $\mathbb{C}_{i-1}\cup C_{i}$ is merged before $\mathbb{C}_{i}\cup C_{i+1}$, 3. 3.

for each $i=1,\dots,k-1$, $|\mathbb{C}_{i}|\geq C_{i+1}$ and the algorithm moves $C_{i+1}$ to the server of $\mathbb{C}_{i}$, 4. 4.

$|C_{1}|<2^{t}$* and $|\mathbb{C}_{2}|=|C_{1}\cup C_{2}|\geq 2^{t}$.*

Note that when considering a doubling decomposition, there will be exactly one majority-vote for the components $\mathbb{C}_{j}$ — the one after $C_{1}$ and $C_{2}$ are merged. Thus, $C$ and all $\mathbb{C}_{j}$, $j\geq 2$, will be assigned to the server that was picked in the majority vote of $C_{1}\cup C_{2}$.

The following lemma shows that doubling decompositions are indeed well-defined. Its proof provides the construction of a doubling decomposition for a given connected component.

Lemma 9**.**

Let $C$ be a connected component. Then there exists a doubling decomposition $(C_{1},\dots,C_{k})$ for $C$.

Proof.

Suppose $(u,v)$ was the last edge which caused the algorithm to set $C=C_{u}\cup C_{v}$. W.l.o.g. assume that $|C_{u}|\leq|C_{v}|$ (in case of ties let $C_{u}$ be the connected component that is moved by the algorithm). Then set $C_{k}=C_{u}$ and set $\mathbb{C}_{k-1}=C_{v}$. Now repeat this procedure for $\mathbb{C}_{k-1}$ in place of $C$ to obtain $C_{k-1}$ and $\mathbb{C}_{k-2}$. Continue this procedure until $C_{1}$ is of appropriate size.

Note that Properties 1 and 2 follow immediately from the above construction. Property 3 follows from the definition of small-to-large steps and the choice of $C_{u}$ above. Property 4 is guaranteed by the stopping criterion of the above recursion. ∎

Lemma 10 will be crucial for the proofs of many upcoming claims in this section. The lemma asserts that when a connected component $C$ is currently assigned to the (say) right server but at the end it will be assigned to the left server, then it must contain relatively many vertices that were initially assigned to the right server.

Lemma 10**.**

Let $C$ be a connected component with $|C|\geq 4$. Suppose that $C$ is currently assigned to server $S_{i}$ and that $C$ will be assigned to server $S_{1-i}$ when the algorithm terminates. Then $C$ contains at least $|C|/4$ vertices which were initially assigned to $S_{i}$.

Proof.

Assume w.l.o.g. that $C$ is currently assigned to the right server and it will be assigned to the left server when the algorithm terminates. We show that at least a $1/4$-fraction of the vertices in $C$ must be black. This implies the lemma.

Let $(C_{1},\dots,C_{k})$ be a doubling decomposition of $C$ which exists by Lemma 9. Observe that $C$ must be assigned to the same server as $C_{1}\cup C_{2}$ after they were merged and after the algorithm processed the majority vote for $C_{1}\cup C_{2}$ (by Properties 3 and 4 of doubling decompositions). Thus, $C_{1}\cup C_{2}$ had a black majority, i.e., it contains at least $|C_{1}\cup C_{2}|/2$ black vertices. Since $|C_{1}\cup C_{2}|\geq|C|/2$, $C$ must contain at least $|C|/4$ black vertices. ∎

Now we bound the augmentation that is used by Algorithm 2.

Lemma 11**.**

The load of both servers is bounded by $n/2+4\Delta$. Hence, Algorithm 2 uses at most $4\Delta$ augmentation.

Proof.

Assume that at some point during the execution of the algorithm the (w.l.o.g.) right server contains more vertices than the left server. We bound the load of the right server.

Recall from Lemma 3 that $\Delta\leq n/4$. We start by considering the case where $\Delta=n/4$. In this case, even moving all $n$ vertices to the right server only causes augmentation $n/2=2\Delta$.

Now consider the case where $\Delta<n/4$. Since $\Delta<n/4$, the initial assignment of $S_{1}$ must contain more vertices from either $V_{0}$ or $V_{1}$. Thus, exactly one of the ground truth components $V_{0}$ and $V_{1}$ must have a black majority (as the algorithm colored all vertices initially assigned to $S_{1}$ black). We assume w.l.o.g. that $V_{1}$ has this black majority. This implies that $V_{1}$ has $n/2-\Delta>n/4>\Delta$ black vertices and $V_{0}$ has $\Delta$ black vertices. Further, as the algorithm proceeds, the vertices from $V_{1}$ must be moved to the right server.

The right server contains at each point a (potentially empty) set of vertices from $V_{0}$ and a (potentially empty) set of vertices from $V_{1}$. For the latter set we use the trivial upper bound of $n/2$, while for the earlier set we give a bound of $\Delta/4$. The lemma follows.

Consider a component $C$ which is on the right server and a subset of $V_{0}$. By Lemma 10, $C$ contains at least $|C|/4$ black vertices.

As there are only $\Delta$ black vertices in the ground truth component $V_{0}$ and each component $C\subseteq V_{0}$ on the right server has at least a $1/4$-fraction of black vertices, it follows that all components on the right server which are subsets of $V_{0}$ can only contain $4\Delta$ vertices. ∎

Having derived the bound for the augmentation, our next goal is to show that the cost paid by the algorithm is bounded by $O(\alpha\Delta\log\Delta)$. We start by bounding the cost paid by the algorithm for each connected component.

The following lemma implies that the algorithm pays nothing for components in which all vertices have the same color.

Lemma 12**.**

Let $C$ be a connected component and suppose all vertices in $C$ have the same color. Then the algorithm has never moved the vertices in $C$.

Proof.

We prove the claim by induction over $s=|C|$.

Let $|C|=s=1$. Then $C$ consists of a single vertex. But the algorithm never moves single vertices unless they become part of a larger connected component. Hence, $C$ is not moved.

Now let $|C|=s+1$. Consider the last edge $(u,v)$ which was inserted that forced the algorithm to merge $C=C_{u}\cup C_{v}$. Since in $C$ all vertices have the same color, all vertices in $C_{u}$ and $C_{v}$ must have the same color. By induction hypothesis, the vertices in $C_{u}$ and $C_{v}$ have never been moved before. Thus, $C_{u}$ and $C_{v}$ must be assigned to the same server. This implies that a small-to-large step would not move $C_{u}$ or $C_{v}$. Further, a majority voting step would not move $C_{u}\cup C_{v}$ since all vertices vote for the server which they are already assigned to. Thus, no vertices in $C$ are moved. ∎

Next, we bound the cost paid for any connected component.

Lemma 13**.**

Let $C$ be a connected component. Then the cost (over the entire execution time of the algorithm) paid for the vertices in $C$ is at most $O(\alpha|C|\log|C|)$.

Proof.

Consider a vertex $u\in C$. We perform the following accounting: we assign a token to $u$ each time when it is reassigned to a server and we show that the number of tokens for $u$ is bounded by $O(\log|C|)$. This implies that the total number of reassignments for the vertices in $C$ is $O(|C|\log|C|)$ and the lemma follows.

First, consider the case where $u$ is moved because it is in a smaller connected component (Line 8). Whenever this happens the size of the connected component containing $u$ at least doubled. This can only happen $O(\log|C|)$ times.

Second, consider the case when $u$ is moved because of a majority vote. A majority vote is performed every time when the size of the component containing $u$ doubled. This can only happen $O(\log|C|)$ times and, hence, this can only add another $O(\log|C|)$ tokens for $u$.

Thus, the total number of tokens assigned to $u$ is $O(\log|C|)$. ∎

Note that Lemma 13 is only useful for components of size at most $O(\Delta$): If we were to apply the lemma to a component $C$ of size $\Theta(n)$ then the cost would only be bounded by $O(\alpha n\log n)$. However, this can be much worse than our desired bound of $O(\alpha\Delta\log\Delta)$ when $\Delta\ll n$. Thus, we need a more fine-grained argument to obtain our goal of showing that the cost paid by Algorithm 2 never exceeds $O(\alpha\Delta\log\Delta)$. To do this, we first prove two technical lemmas.

Lemma 14**.**

Suppose $C$ is a component which is moved from $S_{i}$ to $S_{1-i}$ and the vertices in $C$ are never reassigned after this move. Then $C$ contains at least $|C|/8$ vertices which were initially assigned to $S_{i}$.

Proof.

There are only two possible reasons why $C$ is moved: Either due to a small-to-large step (Line 8) or due to a majority voting step (Line 10). We consider both cases separately.

Case 1: $C$ is moved due to a small-to-large step. Then by Lemma 10, $C$ must contain at least $|C|/4$ vertices which were initially assigned to $S_{i}$.

Case 2: $C$ is moved due to a majority voting step.

First, consider the case when $C$ contains at most 7 vertices. Then at least one vertex was initially assigned to $S_{i}$ (if all vertices had been initially assigned to $S_{1-i}$, they would all have the same color and a majority vote would not move $C$ due to Lemma 12). Thus, at least a $1/7$-fraction of the vertices were initially assigned to $S_{i}$ and the lemma holds.

Second, suppose that $C$ contains at least $8$ vertices. Consider the last edge $(u,v)$ that caused the merge $C=C_{u}\cup C_{v}$. Suppose that the small-to-large step moved $C_{u}$ to the server of $C_{v}$. Note that $C_{v}$ was assigned to $S_{i}$ and $C_{u}$ was moved to $S_{i}$. Now apply Lemma 10 to $C_{v}$. This implies that $C_{v}$ must contain at least $|C_{v}|/4$ vertices that were initially assigned to $S_{i}$. As $|C_{u}|\leq|C_{v}|$, $C$ must contain at least $|C|/8$ vertices that were initially assigned to $S_{i}$. ∎

We are now ready to show that the cost incurred by the majority-voting algorithm never exceeds $O(\alpha\Delta\log\Delta)$.

Lemma 15**.**

The total cost paid by Algorithm 2 is at most $O(\alpha\Delta\log\Delta)$ and the final assignment is a perfect partitioning.

Proof.

When the algorithm finishes, the final assignment must be a perfect partitioning because the connected components were completely revealed. We only need to prove that the cost of the algorithm is $O(\alpha\Delta\log\Delta)$.

Recall that OPT moves exactly $2\Delta$ vertices (Lemma 3). We can assume w.l.o.g. that OPT moves $\Delta$ vertices from $V_{0}$ that were initially assigned to $S_{1}$ to $S_{0}$ and $\Delta$ vertices from $V_{1}$ that were initially assigned to $S_{0}$ to $S_{1}$. We will argue that the cost paid by the algorithm for moving all vertices from $V_{0}$ into the $S_{0}$ will be $O(\alpha\Delta\log\Delta)$; the same will hold for $V_{1}$ and $S_{1}$ symmetrically.

Consider time $T$ during the execution of the algorithm where the following happens. A connected component $C$ is reassigned the left server and $C$ has the following properties: (1) $C$ is a subset of $V_{0}$ and (2) the vertices in $C$ never leave the left server after time $T$. Since each vertex of $V_{0}$ is assigned to the left server when the algorithm terminates, each vertex of $V_{0}$ is contained in a component with the above properties (when a vertex or component is never moved, we set $T=0$). We call a component with the above properties mixed if it contains at least one black vertex. Note that when mixed component $C$ is assigned to the left server, $C$ contains a black vertex and, hence, $C$ must be moved from the right to the left server.

We now bound the cost for mixed components. Let $X$ be the set of all mixed components and let $C\in X$. Since $C$ is mixed, Lemma 14 implies that at least $|C|/8$ vertices of $C$ are black. As the black vertices in mixed components form a partition of the $\Delta$ black vertices in $V_{0}$ moved by OPT , we obtain that the number of black vertices in mixed components is $\Delta$. Thus, the total number of vertices in all mixed components is $\sum_{C\in X}|C|\leq 8\Delta$.

By Lemma 13, the total cost paid for each $C\in X$ until (including) its final move is $O(\alpha|C|\log|C|)$. Since (by assumption) the vertices in $C$ never move between the servers again, their cost never exceeds $O(\alpha|C|\log|C|)$ until the algorithm finishes. Hence, the cost paid by the algorithm for all mixed components is

[TABLE]

Now consider the vertices of $V_{0}$ which are not part of mixed components. These vertices must have been part of components in which all vertices are colored yellow. By Lemma 12, these vertices have never been moved. Thus, they do not incur any additional cost for the algorithm. ∎

Proof of Proposition 7.

Lemma 11 gives the bound for the augmentation used by the algorithm. By Lemma 15 and Lemma 3, Algorithm 2 obtains a competitive ratio of

[TABLE]

3.4 Bringing It All Together: Theorem 2

Proof.

Proof of Theorem 2. Consider the following algorithm: Run the majority-voting algorithm until we have seen all edges or until at some point it tries to exceed the allowed augmentation. In the latter case, compute a perfectly balanced assignment respecting the connected components and start running Algorithm 1 (Section 3.2.2).

To prove the theorem, we distinguish two cases based on $\Delta$.

First, suppose $\Delta<\varepsilon n/4$. By Proposition 7, Algorithm 2 uses at most $4\Delta$ augmentation. Thus, in the current case the augmentation used by Algorithm 2 is bounded by $4\Delta<\varepsilon n$ and it is $O(\log n)$-competitive. This proves the theorem for this case.

Second, suppose $\Delta\geq\varepsilon n/4$. In this case we run Algorithm 2 until it tries to exceed the allowed augmentation; this serves as a certificate that $\Delta\geq\varepsilon n/4$. At this point we switch to Algorithm 1.

When we switch algorithms, Algorithm 2 has paid $O(\alpha n\log n)$, by applying Lemma 13 to each connected component, and then summing over these costs. For switching to the perfectly balanced reassignment, we only need to pay $O(\alpha n)$ once.

By Proposition 6, Algorithm 1 never uses more than $O(n\log n+(\Delta\log n)/\varepsilon)$ vertex moves. Using the bound $\Delta\geq\varepsilon n/4$ and the fact that $\operatorname{OPT}$ pays $2\alpha\Delta$ (Lemma 3), we obtain the desired competitive ratio:

[TABLE]

4 Generalization to Many Servers

We extend our study to the scenario with $\ell$ servers. As we will see, while several concepts introduced for the two server case are still useful, the $\ell$-server case introduces additional challenges. We derive the following main result.

Theorem 16**.**

Given a system with $\ell$ servers each of capacity $(1+\varepsilon)n/\ell$ (i.e., augmentation $\varepsilon n/\ell$), for $\varepsilon\in(0,1/2)$, then there exists an $O((\ell\log n\log\ell)/\varepsilon)$-competitive algorithm.

Our algorithm will be based on a recursive bipartitioning scheme, described in Section 4.1. We will use this bipartitioning scheme to derive a static approximation algorithm of the optimal solution (Section 4.2). Then we provide a recursive version of the majority voting algorithm which we will compare against the approximation algorithm (Section 4.3). In Section 4.4, we analyze the Small–Large–Rebalance algorithm in the $\ell$ server setting and we conclude by proving Theorem 16 in Section 4.5

4.1 The Bipartition Tree

We establish a recursive bipartitioning scheme of the servers which we will be using throughout the rest of this section. All algorithms in this section which use the recursive bipartitioning create such a bipartitioning at the start of the algorithm, before the adversary provides any edge. After that the bipartitioning will never be changed.

We obtain the bipartition scheme by growing a balanced binary tree on a set of $\ell$ leaves, where each leaf corresponds to a server $S_{i}$. We call this tree the bipartition tree and denote it by $\mathcal{T}$.

We denote the internal nodes of $\mathcal{T}$ by $w_{1},\dots,w_{s}$. For an internal node $w_{j}$, we write $T(w_{j})$ to denote the subtree of $T$ which is rooted at $w_{j}$ and we define $S(w_{j})$ to be the set of servers which are leaves in $T(w_{j})$. We further write $V(w_{i})$ to denote the set of vertices which are assigned to the servers in $S(w_{j})$. See Figure 3 for an illustration.

Observe that $\mathcal{T}$ defines a bipartition scheme: let $w$ be an internal node of $\mathcal{T}$ and let $w_{0}$, $w_{1}$ be its children. Then444If $w_{j}$ is a leaf corresponding to server $S$, we set $S(w_{j})=\{S\}$. $S(w_{0})$ and $S(w_{1})$ are disjoint and their union is $S(w)$. Thus, $\mathcal{T}$ implies a bipartition scheme of the servers and internal nodes correspond to bipartition steps.

Note that since $\mathcal{T}$ is a balanced binary tree, there are $\ell-1$ internal nodes in total and each server is contained in at most $\lceil\log\ell\rceil$ subtrees of $T$. Hence, for each server $S_{j}$ there are at most $\lceil\log\ell\rceil$ internal vertices $w$ such that $S_{j}\in S(w)$.

In the following we will refer to the internal nodes in $\mathcal{T}$ as nodes, whereas the vertices $V$ from the graph $G$ are called vertices.

4.2 Offline Approximation Algorithm

We are not aware of a concise characterization of the optimal solution used by OPT (unlike in the two-server case in Section 3.1). Thus, to get a better understanding of the solution obtained by OPT , we provide an offline approximation algorithm, called APPROX , which exploits the previously defined bipartition scheme and which obtains a 2-approximation of the optimal solution. However, unlike the solution obtained by OPT , we allow the approximation algorithm to use unlimited augmentation in each server; its only goal is to move all vertices from the same ground truth components to the same server using few vertex moves.555In this setting, a trivial solution assigns all vertices to the same server at cost $O(\alpha n)$. Later, APPROX will play a role for the design and analysis of our online algorithm.

Intuitively, APPROX traverses the bipartition tree $\mathcal{T}$ top–down and greedily minimizes the number of vertices “moved over” each server bipartition. We now describe the algorithm in more detail.

APPROX is given the sequence of edges $\sigma=(e_{1},\dots,e_{r})$ a priori and it also knows the initial assignment $V_{\operatorname{init}}(S_{0}),\dots,V_{\operatorname{init}}(S_{\ell-1})$ of the vertices to the $\ell$ servers. Using the knowledge about the edges, APPROX starts by computing the connected components of $G$ and obtaining the ground truth components $V_{0},\dots,V_{\ell-1}$.

Now, for each ground truth component $V_{i}$, APPROX does the following. Let $\mathfrak{r}$ be the root of $\mathcal{T}$ and let $w_{0}$ and $w_{1}$ denote its children. Let $A_{ij}=V(w_{j})\cap V_{i}$, $j=0,1$, denote the vertices from $V_{i}$ which are currently assigned to servers in $S(w_{j})$. Define $n_{ij}=|A_{ij}|$ and assume w.l.o.g. that $n_{i0}\geq n_{i1}$. The algorithm marks the vertices from $A_{i1}$ as dirty. Now the algorithm recurses on the subtree $T(w_{0})$ in place of $\mathcal{T}$ and marks more vertices of $V_{i}$ as dirty. The recursion stops when $S(w_{0})$ only contains a single server $S$. Then the algorithm moves all dirty vertices of $V_{i}$ into server $S$.

The pseudocode of APPROX is stated in Algorithm 3.

By overloading notation, we let APPROX denote the cost paid by APPROX . Further, we let $\operatorname{APPROX}_{i}$ denote the cost paid by APPROX to move all vertices from $V_{i}$ to the same server $S$.

We now show that APPROX indeed yields a 2-approximate solution of the cost of the optimal offline algorithm.

Lemma 17**.**

$\operatorname{APPROX}\leq 2\cdot\operatorname{OPT}$.

Proof.

Fix any $i\in\{0,\dots,\ell-1\}$. Let $\operatorname{OPT}_{i}$ denote the cost paid by OPT to move the vertices from $V_{i}$ to the same server. We show that $\operatorname{APPROX}_{i}\leq 2\cdot\operatorname{OPT}_{i}$. This claim implies the lemma since

[TABLE]

Observe that while $\operatorname{APPROX}_{i}$ proceeds, it traverses $\mathcal{T}$ from root $\mathfrak{r}$ to one of the leaves, and at each step, it increases the level of the current internal node by one.

Using the solution of $\operatorname{OPT}_{i}$, we can define a similar traversal of $\mathcal{T}$: Let $\mathfrak{r}$ be the root of $\mathcal{T}$ and let $w_{0}$ and $w_{1}$ be its children. As $\operatorname{OPT}_{i}$ must move all vertices from $V_{i}$ to the same server $S$, $\operatorname{OPT}_{i}$ moves the vertices from $A_{ij}=V(w_{j})\cap V_{i}$ to a server $S$ in $S(w_{1-j})$ for $j\in\{0,1\}$. We call the moved vertices dirty. After this move, $\operatorname{OPT}_{i}$ still needs to process the vertices of $V_{i}$ which were initially assigned to a server in $S(w_{1-j})$ but not to $S$. We can view this as processing $T(w_{j-1})$. Thus, $\operatorname{OPT}_{i}$ traverses $\mathcal{T}$ until the final server $S$ is reached and marks a subset of $V_{i}$ dirty.

The previous paragraphs define to two different traversals of $\mathcal{T}$ and two different sets of dirty vertices. Let $h$ be the smallest level where the two traversals picked different internal nodes in $\mathcal{T}$.

Until level $h-1$, both vertices have marked the same $W$ vertices dirty. At levels $h$ and below, we obtain the following bounds. Let $w$ be the internal node at level $h-1$ that is traversed by both algorithms and let $w_{0}$, $w_{1}$ denote its children at level $h$. Let $n_{ij}=|V(w_{j})\cap V_{i}|$ be defined as in the definition of APPROX . $\operatorname{APPROX}_{i}$ marks at most $|V(w)|=n_{i0}+n_{i1}$ vertices from $V(w)$ as dirty. Since the two traversals split at level $h$ and $\operatorname{APPROX}_{i}$ moves $n_{i1}\leq n_{i0}$ vertices (by definition), $\operatorname{OPT}_{i}$ moves at least $n_{i0}$ vertices.

Recall that for each algorithm, its sets of dirty vertices and its set of moved vertices are identical. Now the following computation proves the claim that $\operatorname{APPROX}_{i}\leq 2\cdot\operatorname{OPT}_{i}$:

[TABLE]

4.3 The Recursive Majority Voting Algorithm

We now describe an algorithm which works efficiently in the setting with $\ell$ servers whenever OPT does not perform too many vertex moves. The algorithm can be viewed as a generalization of Algorithm 2 to $\ell$ servers, by exploiting the previously defined bipartitioning scheme.

4.3.1 The Algorithm

The algorithm consists of two parts: A single global algorithm and multiple local algorithms, one per internal node in $\mathcal{T}$. The global algorithm maintains a recursive bipartitioning scheme (as defined in Section 4.1) and runs a local algorithm on each bipartition. The local algorithms are used to “reduce” the setting with multiple servers to the case with two servers.

We now describe the two parts in more detail and state the pseudocode in Algorithm 4. We write $S_{u}$ to denote the server which vertex $u$ is assigned to.

Global Algorithm.

The global algorithm starts by computing the bipartition tree $\mathcal{T}$. On each internal node $w$ of $\mathcal{T}$, the global algorithm instantiates a local algorithm which we describe below.

Furthermore, the global algorithm iterates over all vertices and does the following for each $v\in V$. The algorithm finds all internal nodes $w_{i}$ such that $v\in V(w_{i})$ and labels $v$ with $w_{i}$. This labelling of the vertices only takes into account the initial assignment of the vertices and will never be changed throughout the running time of the algorithm. For example, if the vertices $u$ and $v$ in Figure 3 are assigned to servers $S_{0}$ and $S_{3}$ in the initial assignment, their labels will be $\{w_{1},w_{2},w_{4}\}$ and $\{w_{1},w_{2},w_{5}\}$, respectively.

When the adversary provides an edge $(u,v)$, the global algorithm does the following. It locates the servers $S_{u}$ and $S_{v}$. If $S_{u}=S_{v}$, the algorithm merges the components and continues with the next edge. If $S_{u}\neq S_{v}$, the global algorithm finds the internal node $w$ in $\mathcal{T}$ which is the lowest common ancestor of $S_{u}$ and $S_{v}$. (For example, in Figure 3 the lowest common ancestor for $u$ and $v$ is $w_{2}$.) Then the global algorithm gives the edge $(u,v)$ to the local algorithm corresponding to $w$.

Local Algorithms.

A local algorithm is run on an internal node $w$ of $\mathcal{T}$. Let $w_{0}$ and $w_{1}$ denote the children of $w$ in $\mathcal{T}$. Note that each local algorithm corresponds to a bipartition step where the servers in $S(w)$ are partitioned into subsets $S(w_{0})$ and $S(w_{1})$.

An instance of the local algorithm only receives edges $(u,v)$ from the global algorithm when (1) their endpoints are assigned to servers $S_{u},S_{v}\in S(w)$ and (2) $S_{u}$ and $S_{v}$ are in different sets of the bipartition, i.e., $S_{u}\in S(w_{j})$ and $S_{v}\in S(w_{1-j})$.

When the global algorithm provides an edge $(u,v)$ with the above properties, the local algorithm locates $C_{u},C_{v}$, $S_{u}$ and $S_{v}$. Assume w.l.o.g. that $|C_{u}|\leq|C_{v}|$. Then $C_{u}$ is moved to $S_{v}$ and $C_{u}$ and $C_{v}$ are merged.666When $C_{u}$ changes its server, all local algorithms corresponding to internal nodes $w$ with $S_{u}\in S(w)$ or $S_{v}\in S(w)$, must be informed about this move. This can be done by recomputing $V(w)$ for each internal node $w$. Note that this is just an internal operation of the data structure and does not incur any cost to the algorithm. As before, we call this a small-to-large step.

Finally, the local algorithm checks whether the new component $C_{u}\cup C_{v}$ has size $n/\ell$ or it surpassed a power of $2$, i.e., it checks if $|C_{u}\cup C_{v}|=n/\ell$ or there exists an $i\in\mathbb{N}$ s.t. $|C_{u}|<2^{i}$, $|C_{v}|<2^{i}$ and $|C_{u}\cup C_{v}|\geq 2^{i}$. If this is the case, the local algorithm triggers a majority voting step for $C_{u}\cup C_{v}$ which we explain next.

Majority Voting Step.

When a local algorithm triggers a majority voting step for a connected component $C$, the algorithm does the following. Let $\mathfrak{r}$ be the root of $\mathcal{T}$ and let $w_{0}$ and $w_{1}$ be the two child nodes of $\mathfrak{r}$. For $j\in\{0,1\}$, let $n_{j}$ denote the number of vertices in $C$ with label $w_{j}$. If $n_{j}\geq n_{1-j}$, the algorithm recurses on $w_{j}$ in place of $w$; else, the algorithm recurses on $w_{1-j}$ in place of $w$. The recursion continues until a leaf in the bipartitioning tree is reached which corresponds to a server $S$. Then the algorithm moves $C$ to $S$.

Note that the above majority voting procedure is very similar to what APPROX does for a single ground truth component $V_{i}$.

Stopping Criterion.

To ensure that the algorithm does not exceed the augmentation of the servers, we add a stopping criterion.

To define the stopping criterion, let $w$ be an internal node of $\mathcal{T}$ with children $w_{0}$ and $w_{1}$. For $j\in\{0,1\}$, we call $w_{j}$ overloaded if $V(w_{j})$ contains at least $\varepsilon n/(\ell\lceil\log\ell\rceil)$ vertices with label $w_{1-j}$.

Intuitively, the condition states that an internal node $w_{j}$ is overloaded when its servers $S(w_{j})$ obtained “many” vertices which were initially assigned to the other side of the bipartition, $S(w_{1-j})$.

The stopping criterion is checked before each component move (i.e., before each small-to-large step and before each majority voting step). It is triggered if the component move would create an assignment in which there exists an overloaded internal node $w$. When the stopping criterion is triggered, the global algorithm and all local algorithms stop and Algorithm 1 is started instead (we show in Section 4.4 that Algorithm 1 also works for $\ell$ servers).

4.3.2 Structural Properties

To obtain a better understanding of the algorithm, we first prove some structural properties about it and defer its cost analysis to Section 4.3.3. We consider the setting where each server has capacity $(1+\varepsilon)n/\ell$ for $\varepsilon\in(0,1/2)$.

In Subsections 4.3.2 and 4.3.3, we only analyze the cost Algorithm 4 without the cost of Algorithm 1. We analyze the cost of Algorithm 1 for $\ell$ servers in Sections 4.4 and 4.5.

We begin by showing that as long as the stopping criterion is not triggered, the vertex assignment created by Algorithm 4 is close to the initial assignment.

Lemma 18**.**

Suppose the stopping criterion is not triggered. Then:

Each server contains at most $\varepsilon n/\ell$ vertices that were not initially assigned to it. 2. 2.

Each server contains at least $(1-\varepsilon)n/\ell$ vertices that were initially assigned to it.

Proof.

Consider any server $S_{j}$. We show that since the stopping criterion is not triggered, $V(S_{j})$ obtains at most $\varepsilon n/(\ell\lceil\log\ell\rceil)$ vertices for each of the $\lceil\log\ell\rceil$ subtrees in $\mathcal{T}$ containing $S_{j}$.

As argued in Section 4.1, there are at most $\lceil\log\ell\rceil$ internal nodes $w$ of $\mathcal{T}$ such that $S_{j}\in S(w)$. Since the stopping criterion is not triggered, no internal node of $\mathcal{T}$ is overloaded.

To prove Part (1), consider an internal node $w$ of $\mathcal{T}$ with $S_{j}\in S(w)$. Let $w_{0},w_{1}$ be the children of $w$ and suppose $S_{j}\in S(w_{r})$. Observe that $V(S_{j})$ can obtain at most $\varepsilon n/(\ell\lceil\log\ell\rceil)$ vertices that were originally assigned to servers in $S(w_{1-r})$ (if it had received more vertices, then $w_{1-r}$ would be overloaded). As there are at most $\lceil\log\ell\rceil$ nodes $w$ with the above property, the number of vertices which were not initially assigned to $S_{j}$ is bounded by $\varepsilon n/\ell$.

Now let us prove Part (2). Consider an internal node $w$ of $\mathcal{T}$ with $S_{j}\in S(w)$. Let $w_{0},w_{1}$ be the children of $w$ and suppose $S_{j}\in S(w_{r})$. Now observe that the servers in $S(w_{1-r})$ can have obtained $\varepsilon n/(\ell\lceil\log\ell\rceil)$ vertices that were originally assigned to $S_{j}$ (if they had received more vertices, then $w_{1-r}$ would be overloaded). As there are at most $\lceil\log\ell\rceil$ nodes $w$ with the above property, it follows that the number of vertices assigned to servers $\{S_{0},\dots,S_{\ell-1}\}\setminus\{S_{j}\}$ that were initially assigned to $S_{j}$ is $\varepsilon n/\ell$. Hence, $S_{j}$ must contain at least $(1-\varepsilon)n/\ell$ vertices that were initially assigned to it. ∎

As a corollary of Lemma 18 we obtain the following lemma.

Lemma 19**.**

(1) As long as the stopping criterion is not triggered, the load of each server is bounded by $(1+\varepsilon)n/\ell$, i.e., Algorithm 4 uses only $\varepsilon n/\ell$ augmentation.

(2) When the stopping criterion is triggered, the augmentation still does not exceed $\varepsilon n/\ell$.

Proof.

Part (1) of the lemma follows immediately from Part (1) of Lemma 18. Let us prove Part (2): The stopping criterion is checked every time before a component is moved. Hence, at the time when the algorithm checks the stopping criterion, the algorithm did not exceed the augmentation bound due to Part (1). If the algorithm triggers the stopping criterion, then the component was not yet moved and the augmentation is still the same as before. ∎

Define the final assignment to be the assignment which is created by Algorithm 4 once it has seen all edges in $G$. We show that the final assignment of the algorithm provides a perfect partitioning if the stopping criterion is not triggered.

Lemma 20**.**

If Algorithm 4 stops and the stopping criterion is not triggered, then the final assignment is a perfect partitioning.

Proof.

By definition of the algorithm, vertices of the same connected component are always assigned to the same server. When the algorithm finishes, all edges of $G$ were revealed and each component has size $n/\ell$. By Lemma 18, the augmentation of each server is at most $\varepsilon n/\ell$. Since $\varepsilon<1/2$, no server can have more than one component assigned. As each component is placed on a server, each component is placed alone on a server. This proves that the algorithm creates a perfect partitioning. ∎

Indeed, we show that the final assignment of Algorithm 4 is not only a perfect partitioning, but it is the same assignment as the one created by APPROX from Section 4.2.

Lemma 21**.**

If Algorithm 4 stops and the stopping criterion is not triggered, Algorithm 4 and APPROX have the same final assignment.

Proof.

By Part (2) of Lemma 18, Algorithm 4 moves at most $\varepsilon n/\ell$ vertices out of each server compared to the initial assignment. Hence, in the final assignment each server must still contain at least $(1-\varepsilon)n/\ell>n/(2\ell)$ vertices from its original assignment since $\varepsilon\in(0,1/2)$. Thus, in the final assignment each server contains more than half of the vertices that were originally assigned to it.

Consider any server $S_{j}$ and let $V_{\operatorname{init}}(S_{j})$ be the set of vertices initially assigned to $S_{j}$. Then there must exist a ground truth component $V_{i}$ with $|V_{i}\cap V_{\operatorname{init}}(S_{j})|\geq n/(2\ell)$. We show that APPROX and Algorithm 4 both assign this component $V_{i}$ to $S_{j}$. This proves the lemma since this claim holds for any $S_{j}$.

First, consider APPROX . Note that at each step of the traversal of $\mathcal{T}$, the majority of the vertices in $V_{i}$ will vote for the internal node containing server $S_{j}$. Hence, APPROX will place $V_{i}$ on $S_{j}$.

Second, consider Algorithm 4. When the algorithm stops, all edges were revealed and the connected components agree with the ground truth components. Now consider the component $C=V_{i}$. When the $C$ grows to size $|C|=n/\ell$, the algorithm performs a majority voting step (by definition of the algorithm). At this point, more than half of the vertices in $C$ were labeled with $S_{j}$ (because more than half of the vertices from $C=V_{i}$ were originally assigned to $S_{j}$). Hence, Algorithm 4 will also place $V_{i}$ on $S_{j}$. ∎

4.3.3 Analysis

The rest of this subsection is devoted to proving the following proposition about Algorithm 4.

Proposition 22**.**

Suppose there are $\ell$ servers and each has capacity $(1+\varepsilon)n/\ell$ for $\varepsilon\in(0,1/2)$, i.e., the augmentation is $\varepsilon n/\ell$. Algorithm 4 has the following properties:

If the stopping criterion is not triggered, the algorithm creates a perfect partitioning, its cost is bounded by $O(\operatorname{OPT}\cdot\log n)$ and at no point during its execution it uses more than $\varepsilon n/\ell$ augmentation. 2. 2.

If the stopping criterion is triggered, the cost of the algorithm is $O(\alpha n\log n)$ plus the cost of Algorithm 1 and the cost of OPT is at least $\Omega(\alpha\varepsilon n/(\ell\log\ell))$.

We prove the proposition at the end of this section. We start by proving a sequence of lemmata and begin by reasoning about the cost paid by Algorithm 4. As shown in Lemma 1 we only need to bound the moving cost paid by Algorithm 4 to bound its total cost.

The following lemma bounds the cost paid for any connected component $C$.

Lemma 23**.**

Let $C$ be a connected component. Then the cost (over the entire execution time of the algorithm) paid for moving the vertices in $C$ is $O(\alpha|C|\log|C|)$.

Proof.

We can use the same accounting argument as in the proof of Lemma 13. That is, we assign a token to a vertex $v$ whenever it is moved. Now, whenever the component $C$ containing $v$ is moved due to a small-to-large step, the size of $C$ doubles. This can only happen $O(\log|C|)$ times. Furthermore, there are only $O(\log|C|)$ majority voting steps involving $u$: Each majority voting step is triggered because $|C|=n/\ell$ or because $|C|$ surpassed a power of $2$; the first event can happen only once and the second event can happen at most $O(\log|C|)$ times. Hence, $v$ will never accumulate more than $O(\log|C|)$ tokens. Since the above arguments apply for each $v\in C$, the total cost paid for moving the vertices in $C$ is bounded by $O(|C|\log|C|)$. ∎

Let $f\colon V\to\{0,\dots,\ell-1\}$ be the function which maps each vertex to its server in the final assignment by Algorithm 4. That is, when Algorithm 4 processed all edges, each $v$ is assigned to $S_{f(v)}$. For a connected component $C$, set $f(C)=f(u)$ for $u\in V$. Note that $f(C)$ is well-defined since all vertices of $C$ are assigned to the same $S_{f(C)}$ when the algorithm terminates.

In the following proofs, we will write $\#{w}({C})$ to denote the number of vertices in a connected component $C$ which are labeled with $w$. We further write $\overline{\#{w}({C})}$ to denote the number of vertices in $C$ which are not labeled with $w$, i.e., $\overline{\#{w}({C})}=|C|-\#{w}({C})$.

Lemma 24 shows that whenever a component $C$ is assigned to a server which is not its final server, it must contain relatively many vertices which were not initially assigned to its final server $S_{f(C)}$.

Lemma 24**.**

Consider any point in the execution of the algorithm at which a connected component $C$ is assigned to server $S\neq S_{f(C)}$. Let $w$ be the lowest common ancestor of $S$ and $S_{f(C)}$ in $\mathcal{T}$ and denote the children of $w$ by $w_{0}$ and $w_{1}$.

If $S_{f(C)}\in S(w_{j})$ for $j\in\{0,1\}$, then:

$C$* contains at least $|C|/4$ vertices which do not have label $w_{j}$, i.e., $\overline{\#{w_{j}}({C})}\geq|C|/4$.* 2. 2.

$C$* contains at least $\overline{\#{w_{j}}({C})}$ vertices which were not initially assigned to $S_{f(C)}$.*

Proof.

To prove Part (1), consider a doubling decomposition $(C_{1},\dots,C_{k})$ of $C$ (see Definition 8); the decomposition exists by Lemma 9 which also applies in the $\ell$ server setting. After $C_{1}$ and $C_{2}$ were merged, Algorithm 4 performed a majority voting step and placed $C_{1}\cup C_{2}$ in a server $S\in S(w_{1-j})$. Thus, $\#{w_{j}}({C_{1}\cup C_{2}})\leq|C_{1}\cup C_{2}|/2$ (otherwise, the majority voting step would have chosen a server in $S(w_{j})$). Since $|C_{1}\cup C_{2}|\geq|C|/2$ and $C_{1}\cup C_{2}\subseteq C$,

[TABLE]

For Part (2) note that each vertex which was initially assigned to $S_{f(C)}$ has label $w_{j}$ (because $S_{f(C)}\in S(w_{j})$ by assumption). ∎

In the following, we show that the cost paid by the algorithm is $O(\operatorname{OPT}\cdot\log n)$ when the stopping criterion is not triggered. We start by showing that when a component is moved for the last time, it contains a large number of vertices which did not originate from the server it is assigned to.

Lemma 25**.**

Let $C$ be a component which is moved to server $S_{f(C)}$ and suppose the vertices of $C$ are never reassigned after this move.777Note that when a small-to-large step is performed, two components are merged due to the corresponding edge insertion. In this case, the component $C$ in the lemma is the component which is being moved (i.e., before merging). Then $C$ contains at least $|C|/8$ vertices which were not assigned to $S_{f(C)}$ in the initial assignment.

Proof.

Note that $C$ is moved due to one of two reasons: Either because of a small-to-large step or because of a majority voting step. We distinguish between these cases.

In case of a small-to-large step, $C$ is assigned to a server $S\neq S_{f(C)}$ before the move. Lemma 24 implies that $C$ contains at least $|C|/4$ vertices which were not originally assigned to $S_{f(C)}$.

Now suppose that $C$ is moved due to a majority voting step. Let $(u,v)$ be the last edge which was inserted and which triggered the majority voting step for $C$. Then Algorithm 4 previously merged components $C_{u}$ and $C_{v}$; suppose w.l.o.g. that $C_{u}$ was moved to $C_{v}$ and $|C_{u}|\leq|C_{v}|$. Prior to the majority voting step, $C$ is assigned to the same server $S\neq S_{f(C)}$ that $C_{v}$ was assigned to before $(u,v)$ was inserted. Hence, we can apply Lemma 24 to $C_{v}$ and obtain that $C_{v}$ contains at least $|C_{v}|/4$ vertices which were not initially assigned to $S_{f(C)}$. Thus, the number of vertices in $C$ which do not originate from $S_{f(C)}$ is at least

[TABLE]

The next lemma considers the cost paid by Algorithm 4 when the stopping criterion is not triggered.

Lemma 26**.**

Suppose there are $\ell$ servers and each has capacity $(1+\varepsilon)n/\ell$ for $\varepsilon\in(0,1/2)$, i.e., the augmentation is $\varepsilon n/\ell$. If the stopping criterion is not triggered and Algorithm 4 stops, then the cost paid by the algorithm is $O(\operatorname{OPT}\cdot\log n)$.

Proof.

Fix some $i\in\{0,\dots,\ell-1\}$. Recall that $\operatorname{APPROX}_{i}$ denotes the cost paid by APPROX to move the vertices from $V_{i}$ to the server $S_{f(V_{i})}$. We show that for $V_{i}$, Algorithm 4 pays $O(\operatorname{APPROX}_{i}\log n)$. The lemma follows from this claim and Lemma 17, since the total cost paid by Algorithm 4 is bounded by

[TABLE]

Consider any ground truth component $V_{i}$ and let $\Delta$ denote the number of vertices $\operatorname{APPROX}_{i}$ moves to server $S_{f(C)}$. Note that as $\operatorname{APPROX}_{i}$ moves $\Delta$ vertices into $S_{f(C)}$, we get $\operatorname{APPROX}_{i}=\alpha\Delta$.

Consider time $T$ of the execution of the algorithm where the following happens. A component $C$ is reassigned to $S_{f(C)}$ and $C$ has the following properties: (1) $C$ is a subset of $V_{i}$ and (2) the vertices in $C$ never leave server $S_{f(C)}$ after time $T$. Since each vertex of $V_{i}$ is assigned to $S_{f(C)}$ when the algorithm terminates, each vertex of $V_{i}$ is contained in a component with the above properties (when a vertex or component is never moved, we set $T=0$). A component $C$ with the above properties is a mixed component if $C$ contains at least one vertex which was not initially assigned to $S_{f(C)}$. Note that when a mixed component $C$ is reassigned to $S_{f(C)}$, $C$ contains at least one vertex which was not initially assigned to $S_{f(C)}$ and, hence, $C$ must be moved from a server $S_{y}$, $y\neq f(C)$, to $S_{f(C)}$.

We bound the cost for mixed components. Let $X$ be the set of all mixed components of $V_{i}$. Recall that Algorithm 4 and APPROX create the same final assignment (Lemma 21). Hence, Algorithm 4 moves the same $\Delta$ vertices from $V_{i}$ into $S_{f(V_{i})}$ as APPROX . Lemma 25 implies that for each $C\in X$ at least $|C|/8$ vertices from $C$ are part of the $\Delta$ vertices moved by APPROX . Thus, the union of all $C\in X$ contains at most $8\Delta$ vertices.

By Lemma 23, Algorithm 4 pays at most $O(\alpha|C|\log|C|)$ for each $C\in X$ over the entire execution. Thus, its total cost is bounded by

[TABLE]

Consider the vertices of $V_{i}$ which are not in mixed components. These vertices must have been part of components in which all vertices were originally assigned to $S_{f(V_{i})}$. By Lemma 12 (which still applies in the $\ell$ server setting), these vertices were never moved. Thus, they do not incur any cost to the algorithm. ∎

Next, we show that when the stopping criterion is triggered, the recursive majority voting algorithm pays $O(n\log n)$ and cost of the solution by OPT is $\Omega(\alpha\varepsilon n/(\ell\log\ell))$.

Lemma 27**.**

When the stopping criterion is triggered, (1) the cost paid by Algorithm 4 is $O(\alpha n\log n)$ and (2) the cost paid by $\operatorname{OPT}$ is $\Omega(\alpha\varepsilon n/(\ell\log\ell))$.

Proof.

Let $Y$ denote the set of all connected components. Part (1) follows from Lemma 23 since the total cost paid by Algorithm 4 is

[TABLE]

Now we prove Part (2). Let $w$ be an internal node of $\mathcal{T}$ with children $w_{0},w_{1}$ and suppose w.l.o.g. that $w_{0}$ is overloaded. Since the stopping criterion is triggered, $V(w_{0})$ contains at least $\varepsilon n/(\ell\log\ell)$ vertices with label $w_{1}$.

Let $X$ be the set of all connected components $C$ with the following properties: $C$ is assigned to a server in $S(w_{0})$ at the time at which the stopping criterion is triggered and $C$ contains at least one vertex which is labeled with $w_{1}$.

To show that OPT performs $\Omega(\varepsilon n/(\ell\log\ell))$ vertex moves, we prove that OPT performs $\Omega(\#{w_{1}}({C}))$ vertex moves for each $C\in X$. Part (2) of the lemma follows since the components in $X$ contain at least $\varepsilon n/(\ell\log\ell)$ vertices with label $w_{1}$ and thus

[TABLE]

We prove that OPT moves at least $\Omega(\#{w_{1}}({C}))$ vertices for each $C\in X$ by distinguishing two cases for $C\in X$. We define $g$ as the function which maps $C\in X$ to the server it is assigned to in the solution of OPT , i.e., OPT assigns $C\in X$ to server $S_{g(C)}$.

Case 1: $S_{g(C)}\not\in S(w_{1})$, i.e., in the final assignment of OPT , the vertices in $C$ are assigned to $S_{g(C)}\not\in S(w_{1})$. Then OPT must perform at least $\#{w_{1}}({C})$ vertex moves because it must move all $w_{1}$-labeled vertices of $C$ from their initial server in $S(w_{1})$ to $S_{g(C)}\not\in S(w_{1})$.

Case 2: $S_{g(C)}\in S(w_{1})$, i.e., in the final solution by OPT , the vertices in $C$ are assigned to a server $S_{g(C)}\in S(w_{1})$. We show that $C$ contains at least $|C|/4$ vertices without label $w_{1}$. This implies the claim since OPT must move at least $\overline{\#{w_{1}}({C})}\geq|C|/4$ vertices from servers not in $S(w_{1})$ to $S_{g(C)}\in S(w_{1})$.

Consider a doubling decomposition $(C_{1},\dots,C_{k})$ of $C$ (which exists by Lemma 9). After $C_{1}$ and $C_{2}$ were merged, the algorithm performed a majority voting step and placed $C_{1}\cup C_{2}$ in a server in $S(w_{0})$. Thus, $\#{w_{1}}({C_{1}\cup C_{2}})\leq|C_{1}\cup C_{2}|/2$ (otherwise, the majority voting step would place $C_{1}\cup C_{2}$ in a server in $S(w_{1})$). Hence, $\overline{\#{w_{1}}({C_{1}\cup C_{2}})}=|C_{1}\cup C_{2}|-\#{w_{1}}({C_{1}\cup C_{2}})\geq|C_{1}\cup C_{2}|/2$. Since $|C_{1}\cup C_{2}|\geq|C|/2$, we get $\overline{\#{w_{1}}({C})}\geq|C|/4$. ∎

Proof of Proposition 22..

The first statement of the proposition is implied by Lemmas 20 (perfect partitioning), 26 (total cost) and 19 (small augmentation). The second statement is proved in Lemma 27 (guarantees when stopping criterion is triggered). ∎

4.4 Small–Large–Rebalance Algorithm for Many Servers

To obtain an efficient algorithm in cases where OPT moves many vertices, we reuse the Algorithm 1 from Section 3.2.2. Note that Algorithm 1 also works with $\ell$ servers because it did not use the fact that there are only two servers. In the setting with $\ell$ servers, we obtain the following result.

Proposition 28**.**

Suppose that all servers have capacity $(1+\varepsilon)n/\ell$ for $\varepsilon>0$, i.e., the augmentation is $\varepsilon n/\ell$. Then the cost paid by the more efficient version of Algorithm 1 is $O(\alpha n\log n+(\operatorname{OPT}\cdot\ell\log n)/\varepsilon)$.

Proof.

The proof of the lemma is almost the same as the proof of Proposition 6. The only difference is that we need to bound the number of rebalance operations differently.

The number of vertex moves performed by the algorithm which always moves the smaller connected component to the server of the larger connected component is $O(n\log n)$ and, hence, it incurs cost $O(\alpha n\log n)$. Now, whenever a server exceeds its capacity, the algorithm must have moved at least $\varepsilon n/\ell$ vertices. This can only happen $O(\ell\log n/\varepsilon)$ times. By the same arguments as in the proof of Proposition 6, each rebalancing operations costs $O(\operatorname{OPT})$. Hence, the cost for all rebalancing steps is bounded by $O(\operatorname{OPT}\cdot\ell\log n/\varepsilon)$. ∎

We should point out that as in Lemma 5, we could also do the repartitioning step of Algorithm 1 by taking any perfectly balanced assignment respecting the connected components. In the analysis this would incur $\Theta(n)$ vertex moves for each such step and, hence, yield an algorithm with $O((n\ell\log n)/\varepsilon)$ vertex moves in total. However, unlike in the two-server case, finding a perfectly balanced assignment respecting the connected components is an NP-hard problem. Nonetheless, the problem can be solved approximately in polynomial time at the cost of a constant factor in the competitive ratio. We discuss this in further detail in Section 5.2.2.

4.5 Bringing It All Together: Theorem 16

Proof of Theorem 16..

Consider the algorithm which first runs Algorithm 4 until the stopping criterion is triggered and then switches to the Algorithm 1 from Section 4.4.

If the stopping criterion of the Algorithm 4 is not triggered, then by Proposition 22 the cost of the algorithm is $O(\operatorname{OPT}\cdot\log n)$. Thus, it is $O(\log n)$-competitive.

If the stopping criterion is triggered, then Algorithm 4 pays $O(\alpha n\log n)$ by Proposition 22 and the cost of OPT is $\Omega(\varepsilon n/(\ell\log\ell))$. Furthermore, the cost of Algorithm 1 is $O(\alpha n\log n+(\operatorname{OPT}\cdot\ell\log n)/\varepsilon)$ by Proposition 28. Hence, we obtain the following competitive ratio:

[TABLE]

5 Distributed and Fast Algorithms

In this section we show how the algorithms from Section 4 can be implemented in a distributed setting (Section 5.1) and how they need to be modified to work in polynomial time at the cost of a slightly worse competitive ratio (Section 5.2).

We should point out that even though we discuss the distributed and polynomial time versions of the algorithms separately, they can easily be combined to obtain a distributed algorithm with polynomial computation time.

5.1 Distributed Algorithm

While in Section 4 we presented algorithms in a centralized model of computation, we now show how Algorithms 1 and 4 can be implemented in a distributed model of computation. For realistic parameter settings, the network traffic caused by our distributed algorithms does not increase (asymptotically) compared to the traffic caused by moving around the vertices between the servers.

In our distributed model of computation we assume that all servers have access to: (1) the number of servers $\ell$, (2) the ID of the root server $S_{0}$, (3) a shared clock, and (4) all-to-all communication.

When computing the network traffic, we will asymptotically count the number of messages sent by the algorithms and we further assume that each message contains $\Theta(\log n)$ bits. For the sake of simplicity we assume that moving a vertex from one server to another incurs cost $\alpha=\Theta(\log n)$.888Note that this is a realistic assumption since in order to move a vertex, a server must send the ID of a vertex to another server. Sending the ID of the vertex requires $\Theta(\log n)$ bits.

Because of this simplifying assumption we do not have to distinguish between the number of messages sent by the algorithm and the number of messages used for moving algorithms.

In this distributed model of computation, we obtain the following main result for the distributed versions of the algorithms.

Theorem 29**.**

Consider a system with $\ell$ servers each of capacity $(1+\varepsilon)n/\ell$ (i.e., augmentation $\varepsilon n/\ell$) for $\varepsilon\in(0,1/2)$. Let $M$ be the number of vertex moves performed by OPT .

Then there exists a distributed $O((\ell\log n\log\ell)/\varepsilon)$-competitive algorithm which sends

$O(M\log n)$* messages if $M=O(\varepsilon n/(\ell\log\ell))$,* 2. 2.

$O((\ell^{2}\log n)/\varepsilon+n\log n+(\operatorname{OPT}\cdot\ell\log n)/\varepsilon)$* messages if $M=\Omega(\varepsilon n/(\ell\log\ell))$.*

In particular, if $\ell=O(\sqrt{\varepsilon n})$, then the algorithm’s communication cost does not exceed its cost for moving vertices.

We show for Algorithm 4 (Section 5.1.1) and for Algorithm 1 (Section 5.1.2) individually how they can be implemented distributedly. After that we prove Theorem 29 in Section 5.1.3.

5.1.1 Making Algorithm 4 Distributed

We start by considering the distributed implementation of Algorithm 4 and obtain the following result.

Lemma 30**.**

Algorithm 4 can be implemented in a distributed model of computation such that the guarantees from Proposition 22 still hold. Furthermore, if OPT performs $M$ vertex moves, then we additionally have the following two properties:

If the stopping criterion is not triggered and the algorithm terminates, then the algorithm sent $O(M\log n)$ messages. 2. 2.

If the stopping criterion was triggered, the algorithm sent $O(n\log n)$ messages.

Proof.

We start by presenting the necessary modifications to the algorithm and analyze the number of sent messages at the end of the proof.

Let us start by observing that each server can maintain a local representation of the bipartition tree $\mathcal{T}$: Since the number of servers $\ell$ is known to all servers and $\mathcal{T}$ does not depend on any other quantity, each server can compute $\mathcal{T}$ locally. Next, the data structure stores for each vertex its ID (requiring $O(\log n)$ bits) and the ID $j$ of the server $S_{j}$ it was initially assigned to (requiring $O(\log\ell)$ bits). Thus, the data structure uses $O(\log n)$ bits of storage for each vertex. In other words, it takes $O(1)$ messages to move a vertex between different servers.

Next, we provide the modifications for checking the stopping criterion, small-to-large steps and for majority voting steps.

Before the algorithm moves a component $C$ from server $S$ to server $S^{\prime}$, $S$ and $S^{\prime}$ need to check whether the move would trigger the stopping criterion. To do so, $S$ and $S^{\prime}$ do the following. First, $S$ asks $S^{\prime}$ for its ID using $O(1)$ messages. Second, $S$ distinguishes between two cases: (1) $C$ contains at most $\lceil\log\ell\rceil$ vertices. Then for each vertex $v\in C$, $S$ sends a message to $S^{\prime}$ containing the ID of the server $v$ was initially assigned to. This requires $O(|C|)$ messages. (2) $C$ contains more than $\lceil\log\ell\rceil$ vertices. Then $S$ locally computes all internal nodes $w$ of the bipartition tree $\mathcal{T}$ which contain $S^{\prime}$ as a leaf. For each such node $w$, let $\bar{w}$ be the sibling of $w$ in $\mathcal{T}$. Now for each $w$, $S$ computes the number of vertices in $C$ which were initially assigned to a server in $S(\bar{w})$. Then $S$ sends these values to $S^{\prime}$ using $O(\log\ell)$ messages. Note that in both cases the algorithm does not send more than $O(|C|)$ messages and these messages can be charged to the moving cost of $C$ (which requires $\Omega(|C|)$ messages) which happens after the checking of the stopping criterion. Third, $S^{\prime}$ receives the messages from $S$ and checks locally whether receiving $C$ would trigger the stopping criterion. If the stopping criterion is not triggered, $S^{\prime}$ tells $S$ to start moving $C$. If the stopping criterion is triggered, $S$ sends a message to the root server $S_{0}$ about this event. Then $S_{0}$ informs all other servers about switching to Algorithm 1. This requires $O(\ell)=O(n\log n)$ messages.

Now suppose the algorithm performs a small-to-large step and the stopping criterion was previously checked and not triggered. In this case, no modifications are necessary: The component $C$ can just be sent from one server to the other at the cost of $O(|C|)$ messages (since each vertex in $C$ can be sent using $O(1)$ messages).

Now suppose a server $S$ needs to perform a majority voting step for a component $C$. First, observe that $S$ can locally decide whether a majority voting step is necessary for $C$ since it must only check the size of $C$. Second, when a majority voting step is necessary, $S$ can locally compute which server $S^{\prime}$ will be the recipient of $C$: For each vertex $v\in C$, $S$ knows which server $v$ was initially assigned to. Hence, for each $v$, $S$ can compute the labels of $v$ w.r.t. the bipartitioning scheme from Section 4.1 locally. Since $S$ also knows $\mathcal{T}$, $S$ can compute to which server $S^{\prime}$ the component $C$ should be moved to. These operations do not require any communication between the servers.

To conclude the proof of the lemma, observe that the distributed algorithm performs exactly as many vertex moves as the centralized algorithm. Hence, the guarantees from Proposition 22 still hold. Next, we analyze the number of messages sent by the algorithm. A small-to-large step moving a component $C$ requires $O(|C|)$ messages. Checking the stopping criterion before moving a component $C$ requires another $O(|C|)$ messages. Checking whether a majority voting step is necessary requires no communication at all. Hence, the number of messages used by the algorithm is linear in its number of vertex moves. Thus, Proposition 22 implies the two additional properties which are claimed in the statement of the lemma. ∎

5.1.2 Making Algorithm 1 Distributed

For the distributed implementation of Algorithm 1 we obtain the following result.

Lemma 31**.**

Algorithm 1 can be implemented in a distributed model of computation such that the guarantees from Proposition 28 still hold. Furthermore, if OPT performs $M$ vertex moves, then the algorithm sends at most $O((\ell^{2}\log n)/\varepsilon+n\log n+(M\ell\log n)/\varepsilon)$ messages.

Proof.

We start by stating which modifications need to be made to make Algorithm 1 distributed.

First, suppose that Algorithm 1 performs a small-to-large step moving a component $C$ and that this move does not make any server exceed its capacity. In this case, no modifications are necessary and the number of messages sent is $O(|C|)$ as we have seen in the proof of Lemma 30.

Second, suppose that a small-to-large step wants to move component $C$ to server $S$ which would cause $S$ to exceed its capacity. Then the algorithm performs the following operations:

$S$ informs the root server $S_{0}$ that a rebuild is required. 2. 2.

$S_{0}$ asks all $\ell$ servers to send the edges that were inserted and caused the merge of two connected components since the last rebuild. The servers send of all these edges together with the timestamps when they were inserted. 3. 3.

$S_{0}$ locally simulates the whole system from the beginning and obtains knowledge about all connected components and which servers they are assigned to. 4. 4.

$S_{0}$ tells all other servers $S_{j}$ which components need to be moved and all servers perform the necessary moves.

Since the distributed algorithm performs exactly the same vertex moves as the centralized algorithm, the distributed algorithm is correct and provides the same guarantees as provided in Proposition 28. We only need to analyze how many messages the algorithm sends. To do so, we analyze each step separately.

Every time Step 1 is performed, it requires $O(1)$ messages. As there are $O((\ell\log n)/\varepsilon)$ rebuilds in total, Step 1 sends $O((\ell\log n)/\varepsilon)$ messages in total.

To bound the number of messages sent in Step 2, recall that in total there are only $O(n)$ edges which merge connected components. Hence, sending these edges requires $O(n)$ messages. Furthermore, when a server did not obtain an edge merging two connected components between two rebuilds, it can inform $S_{0}$ about this in $O(1)$ messages. As this can be the case for at most $\ell$ servers and since there are $O((\ell\log n)/\varepsilon)$ rebuilds, at most $O((\ell^{2}\log n)/\varepsilon)$ messages are sent when servers did not receive new edges.

In Step 3, $S_{0}$ locally simulates the system. This does not incur any network traffic.

Now consider Step 4. During a rebuild, the number of components which the algorithm needs to reassign is trivially bounded by the number of vertex moves performed during the rebuild. Thus, Proposition 28 implies that only $O(n\log n+(\operatorname{OPT}\cdot\ell\log n)/\varepsilon)$ messages are required for all invocations of Step 4.

In total, we obtain that the algorithm sends at most $O((\ell^{2}\log n)/\varepsilon)+n\log n+(M\ell\log n)/\varepsilon)$ messages, where $M$ is the number of vertices moved by OPT . ∎

5.1.3 Proof of Theorem 29

To prove the claim about the competitive ratio of the algorithm observe that the distributed algorithm performs exactly the same vertex moves as the centralized algorithm. Hence, the cost paid by both algorithms is the same and the distributed algorithm has the same competitive ratio as the centralized algorithm in Theorem 16.

The claim about the number of messages sent by the algorithm follows from Lemma 30 and Lemma 31 and summing over the number of messages.

To prove the last claim of the theorem, we distinguish two cases. If the stopping criterion was not triggered, then the claim holds by Lemma 30. If the stopping criterion was triggered, then if $\ell=O(\sqrt{\varepsilon n})$, we obtain that the total number of messages is

[TABLE]

which is exactly the number of vertices moved by Algorithm 1. ∎

5.2 Fast Algorithms

In this section, we discuss the computational challenges when computing perfectly balanced assignments. These computational problems occur when Algorithm 1 performs rebalancing steps (see Section 3.2 and Section 4.4). So far, we were only concerned with algorithms which try to minimize the vertex moves while using potentially exponential running time. We now consider polynomial time algorithms. The only step where our algorithms might use exponential time is during rebalancing. Thus we show next how to perform the rebalancing operations in polynomial time. In the case of $\ell>2$ servers, our polynomial time algorithms perform slightly more vertex moves than the exponential time algorithms.

We discuss the two server case which can be solved optimally in polynomial time in Section 5.2.1. In Section 5.2.2, we argue that in the general case with $\ell>2$ servers this problem is NP-hard. We resolve this issue in Section 5.2.3 by computing approximately balanced assignments in polynomial time.

5.2.1 Computing Perfectly Balanced Assignments for Two Servers

We consider computing a perfectly balanced assignment respecting the connected components for two servers. Specifically, we provide a dynamic program which can find such an assignment in polynomial time.

The dynamic program works as follows. Suppose $C_{1},\dots,C_{q}$ are the connected components assigned to the two servers. Now let $k_{i}=|C_{i}|$ for $i=1,\dots,q$. We create a set $\mathcal{S}$ consisting of integers with the following property: Each number $s\in\mathcal{S}$ corresponds to a set of connected components $\mathbb{C}$ such that $|\bigcup_{C\in\mathbb{C}}C|=s$. That is, whenever $s\in\mathcal{S}$, there exists a set of connected components which together contain $s$ vertices. For each $s\in\mathcal{S}$, the algorithm maintains a set of connected components explicitly. We denote the components corresponding to value $s\in\mathcal{S}$ by $\mathit{components}(S)$.

At the beginning of a rebalancing step, the algorithm sets $\mathcal{S}=\{0\}$. The connected component corresponding to value [math] is simply the empty set of vertices, i.e., $\mathit{components}(0)=\emptyset$. For $i=1,\dots,q$ the algorithm does the following. Iterate over all $s\in\mathcal{S}$ and over all components and add $s+k_{i}$ to $\mathcal{S}$ if $s+k_{i}\not\in\mathcal{S}$ and $C_{i}\not\in\mathit{components}(S)$. Whenever a new value $s+k_{i}$ is added to $\mathcal{S}$, set $\mathit{components}(s+k_{i})=\mathit{components}(s)\cup\{C_{i}\}$.

As soon as the value $n/2$ is added to $\mathcal{S}$, the dynamic program stops and assigns all vertices in $\mathit{components}(n/2)$ to the left server and all remaining vertices to the right server.

The correctness of the above algorithm is clear by construction. We only need to show that it finishes in polynomial time.

Note that the above dynamic program runs in time $O(q|\mathcal{S}|)$. Now observe that $q$ is bounded by $n$ since there are at most $n$ connected components. Furthermore, for each subset $\mathbb{C}\subseteq\{C_{1},\dots,C_{q}\}$, we have that $\sum_{C\in\mathbb{C}}|C|\leq n$ (because the components in $\mathbb{C}$ cannot contain more than $n$ vertices). Thus, $|\mathcal{S}|\leq n+1$ because each value $s\in\mathcal{S}$ corresponds to a subset of components $\mathbb{C}\subseteq\{C_{1},\dots,C_{q}\}$ and each value $s\in\{0,\dots,n\}$ is only added once to $\mathcal{S}$. Hence, the algorithm runs in time $O(q|\mathcal{S}|)=O(n^{2})$.

5.2.2 Computing Perfectly Balanced Assignments for Many Servers

We consider computing a perfectly balanced assignment respecting the connected components for $\ell$ servers.

Let $C_{1},\dots,C_{q}$ be the connected components which are assigned to the $\ell$ servers. To find a perfectly balanced assignment respecting the connected components, we need to find a partition of the set $\mathcal{S}=\{|C_{1}|,\dots,|C_{q}|\}$ into $\ell$ subsets $\mathcal{S}_{1},\dots,\mathcal{S}_{\ell}$ such that for each subset $\mathcal{S}_{i}$ we have that $\sum_{s\in\mathcal{S}_{i}}s=n/\ell$.

Unfortunately, the above problem is known to be NP-complete, see, e.g., the result about multi-processor scheduling in Garey and Johnson [28]. However, since we prove our results in the online model of computation, which allows unlimited computational power, the algorithm can solve this NP-complete problem. We note that this problem has also been studied in practice, see, e.g., Schreiber et al. [48] and references therein.

See Section 5.2.3 for how this problem can be solved approximately at the cost of a constant in the competitive ratio of the algorithm.

5.2.3 Computing Approximately Balanced Assignments for Many Servers

Previously we have we seen that perfectly balanced assignments for $\ell$ servers cannot be computed in polynomial time unless $\textsf{P}=\textsf{NP}$ (Section 5.2.2). Thus, we now consider computing approximately balanced assignments for $\ell$ servers which is sufficient for our purpose: Let $\varepsilon^{\prime}>0$ be a constant. An assignment is $(1+\varepsilon^{\prime})$-approximately balanced if each server has load at most $(1+\varepsilon^{\prime})n/\ell$. Using this definition, we obtain the following result.

Proposition 32**.**

Let $\varepsilon>\varepsilon^{\prime}>0$ be constants and suppose each server has capacity $(1+\varepsilon)n/\ell$. Then a $(1+\varepsilon^{\prime})$-approximately balanced assignment for $\ell$ servers can be computed in polynomial time.

Using the proposition (which we prove at the end of the subsection), we obtain a polynomial time algorithm with a slightly worse competitive ratio than that of Theorem 16.

Theorem 33**.**

Given a system with $\ell$ servers each of capacity $(1+\varepsilon)n/\ell$, for constant $\varepsilon\in(0,1/2)$, then there exists an $O((\ell^{2}\log n\log\ell)/\varepsilon^{2})$-competitive algorithm which runs in polynomial time.

Proof.

First, observe that Algorithm 4 runs in polynomial time. Thus, the result of Proposition 22 also holds for polynomial time algorithms.

Second, consider a modification of Algorithm 1 where at each rebalancing step we compute a $(1+\varepsilon^{\prime})$-approximately balanced assignment for $\varepsilon^{\prime}=\varepsilon/2$. Such an approximately balanced assignment can be computed in polynomial time due to Proposition 32. Thus, the modified algorithm runs in polynomial time.

Observe that now all steps of the resulting algorithm can be computed in polynomial time. It is left to bound the competitive ratio of the modified algorithm.

We start by bounding the cost paid by the modified version of Algorithm 1. Note that each approximate rebalancing step incurs cost at most $O(\alpha n)$; recall that $\alpha$ denotes the cost for moving a vertex to a different server. Now we bound the number of approximate rebalancing steps. Recall from Lemma 4 that the number of vertex moves due to small-to-large steps is at most $O(n\log n)$. Now whenever a new approximately balanced assignment is computed, the small-to-large steps must have moved at least $\Omega((\varepsilon-\varepsilon^{\prime})n/\ell)$ vertices to exceed the capacity of one of the servers. Thus, the total number of approximate rebalancing operations is bounded by $O((\ell\log n)/(\varepsilon-\varepsilon^{\prime}))$ and, hence, the total cost of Algorithm 1 with approximate rebalancing steps is bounded by $O((\alpha n\ell\log n)/(\varepsilon-\varepsilon^{\prime}))$.

Altogether, we obtain the following competitive ratio by following the steps from the proof of Theorem 16 (Section 4.5):

[TABLE]

where in the last step we used that $\varepsilon-\varepsilon^{\prime}=\varepsilon/2$. ∎

To prove Proposition 32, we consider the makespan minimization problem in which there are $k$ jobs with processing times $p_{1},\dots,p_{k}$ which must be assigned to $\ell$ identical machines. Given an assignment of the jobs to the machines, the maximum running time time of any machine is called the makespan. The goal is to find an assignment of the jobs to the machines which minimizes the makespan.

The makespan minimization problem is known to be NP-hard but Hochbaum and Shmoys [31] presented a polynomial time approximation scheme (PTAS).

Lemma 34** (Hochbaum and Shmoys [31]).**

Let $\varepsilon^{\prime}>0$ be a constant. Then there exists an algorithm which computes a $(1+\varepsilon^{\prime})$-approximate solution for the makespan minimization problem in polynomial time.

Using the result from the lemma we can prove Proposition 32.

Proof of Proposition 32.

Suppose the system currently contains connected components $C_{1},\dots,C_{k}$. We consider these connected components as the jobs of the makespan minimization problem with processing times $p_{i}=|C_{i}|$ for $i=1,\dots,k$. The machines correspond to the $\ell$ servers.

Note that the optimal solution for the instance of the makespan minimization problem is $n/\ell$: Since we have made the assumption that in the final assignment all servers have load exactly $n/\ell$, there must exist a perfectly balanced assignment from the components $C_{i}$ to the servers $S_{i}$. In other words, there exists an assignment of the jobs to the machines such that each machine has running time $n/\ell$ and, hence, the optimal makespan is $n/\ell$.

By running the algorithm from Lemma 34, we obtain a $(1+\varepsilon^{\prime})$-approximate solution for the makespan minimization problem. Since the optimal solution for this problem is $n/\ell$, each machine has load at most $(1+\varepsilon^{\prime})n/\ell$ in the solution returned by the algorithm from Lemma 34. Assigning the components $C_{i}$ to the servers in exactly the same way as the corresponding jobs are assigned to the corresponding machines, we obtain a $(1+\varepsilon^{\prime})$-approximately balanced assignment in polynomial time. ∎

6 Lower Bounds

To study the optimality of our algorithms, we derive bounds on the competitive ratios which can be achieved by any deterministic online algorithm.

The following theorem provides a lower bound of $\Omega(1/\varepsilon+\log n)$. The lower bound has the following two main consequences: (1) If an algorithm is only allowed to use constant augmentation (i.e., servers of capacity $n/\ell+O(1)$), then the lower bound implies that any algorithm must have a competitive ratio of $\Omega(n)$.999To obtain servers of capacity $n/\ell+O(1)$, we must set $\varepsilon=O(1)/n$. (2) The lower bound holds even in the setting in which there are only two servers. Thus, the algorithm from Section 3 for the two server setting is close to optimal (up to a $O(\min\{1/\varepsilon,\log n\})$ factor) and the generalized algorithm from Section 4 is optimal up to a $O(\ell\log\ell\min\{1/\varepsilon,\log n\})$ factor.

Theorem 35**.**

Suppose there are two servers of capacity $(1+\varepsilon)n/2$ for $\varepsilon\leq 0.98$. Then any deterministic online algorithm must have a competitive ratio of $\Omega(1/\varepsilon+\log n)$.

To prove the theorem, we show in Section 6.1 that there exist input sequences such that either an algorithm always assigns vertices of the same connected component to the same server or it has prohibitively high cost. Using this fact, we prove our concrete lower bounds in Section 6.2.

6.1 Assigning Connected Components to Servers

In this subsection, we give an important reduction which will be useful to derive the lower bounds in the next subsection (Section 6.2). This reduction lets us assume that every competitive algorithm will always assign vertices of the same connected component to the same server.

More concretely, we show that every sequence of edges $\sigma$ can be manipulated to a new edge sequence $\sigma^{\prime}$ such that: (1) $\sigma$ reveals the same edges as $\sigma^{\prime}$ and (2) on input $\sigma^{\prime}$, every algorithm either moves the vertices of the same connected components to the same server, or has prohibitively high cost and, hence, cannot be competitive.

We first prove the following technical lemma.

Lemma 36**.**

Consider a sequence $\sigma$ which reveals the edges $\emptyset\neq E^{*}\subseteq E$. Let $C_{1},\dots,C_{q}$ be the connected components induced by $E^{*}$.

Then for each initial assignment there exists an input sequence $\sigma^{\prime}$ consisting only of edges in $E^{*}$ such that either (1) at some point during the input sequence the algorithm assigns all vertices from each $C_{i}$ to the same $S_{j}$ or (2) the cost of the algorithm is at least $\Omega(\alpha n^{3})$.

Proof.

We will construct an input sequence $\sigma^{\prime}$ provided by the adversary such that either Property (1) or Property (2) must hold.

Consider an arbitrary initial assignment and pick the ground truth components $V_{i}$ such that they do not coincide with the initial assignment of the vertices to the servers, i.e., $V_{i}\neq V_{\operatorname{init}}(S_{j})$ for all $i,j$. Let $E^{*}=\{e_{1}^{\prime},\dots,e_{t}^{\prime}\}$ be the edges revealed by the adversary and suppose that $E^{*}$ contains at least one edge $(u,v)$ such that $u$ and $v$ are assigned to different servers in the initial assignment.

Now consider the input sequence $\sigma^{\prime}=(e_{1},\dots,e_{r})$ with $r=\lceil\alpha n^{3}t\rceil$ which consists of the edges $(e_{1}^{\prime},\dots,e_{t}^{\prime})$ in $E^{*}$ concatenated $\lceil\alpha n^{3}\rceil$ times.

Suppose that while running the algorithm there always exists a $C_{i}$ such that not all vertices from $C_{i}$ are assigned to the same server $S_{j}$, i.e., Claim (1) does not apply. We show that then Claim (2) must apply.

Consider the state of the algorithm prior to a single subsequence containing the edges $(e_{1}^{\prime},\dots,e_{t}^{\prime})$. By assumption at least one edge $e_{i}^{\prime}$ must be between two vertices from different servers. Now the algorithm must either pay $1$ for communication along this edge or it must move one of the edge’s endpoints at the cost of $\alpha$ to avoid paying for communication along this edge. Thus, the algorithm must pay at least $\Omega(1)$ for the subsequence $(e_{1}^{\prime},\dots,e_{t}^{\prime})$.

As there are $\lceil\alpha n^{3}\rceil$ such subsequences, the algorithm must pay at least $\Omega(\alpha n^{3})$ in total. ∎

As we will see, the lemma essentially allows us to assume that every algorithm which obtains an edge between vertices on different clusters, must move their connected components to the same cluster. That is, given an input sequence $\sigma$, in our lower bound proof, we can employ Lemma 36 to obtain an input sequence $\sigma^{\prime}$ which does not reveal any additional edges and which forces every algorithm to have Property (1) or Property (2).

Now observe that if an algorithm has Property (2), since the cost of OPT are always bounded by $O(\alpha n)$ (OPT moves each vertex at most once), the algorithm cannot be competitive: the competitive ratio must be at least $\Omega(n^{2})$, much higher than the competitive ratios derived in this paper. Hence, in the following we can assume that every algorithm with a competitive ratio better than $\Omega(n^{2})$ must satisfy Property (1) of Lemma 36.

6.2 Lower Bound Proofs

In this subsection, we prove Theorem 35 by proving two different lower bounds: The first lower bound asserts a competitive ratio of $\Omega(1/\varepsilon)$ and the second lower bounds asserts a competitive ratio of $\Omega(\log n)$.

In the lower bound constructions we heavily exploit that we provide hard instances against deterministic algorithms, i.e., we will rely on the fact that at each point in time the adversary knows exactly which assignment the online algorithm created.

Furthermore, we assume that after each edge which was provided by the adversary, the algorithm creates an assignment such that all vertices of the same connected component are assigned to the same server. This assumption is admissible by the discussion in Section 6.1.

We start by proving the lower bound of $\Omega(1/\varepsilon)$.

Lemma 37**.**

Consider the setting with two servers which both have capacity $(1+\varepsilon)n/2$ for $\varepsilon>0$.

Then for each deterministic online algorithm ON there exists an input sequence $\sigma$ such that the cost of ON is $\Omega(\alpha n)$ and the cost paid by OPT is $O(\alpha\varepsilon n)$. Thus, the competitive ratio of every online algorithm is $\Omega(1/\varepsilon)$.

Proof.

Choose an arbitrary initial assignment of $n$ vertices to the $\ell$ servers. Let $K=\varepsilon n/2$ denote the allowed augmentation of the servers. The initial assignment is as follows. In the left server, there are $q=n/(2(K+1))$ connected components $C_{1},\dots,C_{q}$ of size $K+1$. On the right server, we build one connected component of size $K+1$ denoted $C$ and one large connected component of size $n-K-1$ denoted $C^{\prime}$. First, the adversary provides all edges of these connected components at no cost to the algorithm.

Then the adversary inserts an edge from a vertex in $C_{1}$ to a vertex in $C$. Since $C_{1}$ has size $K+1$ and the right server currently has $n/2$ vertices, the algorithm cannot move $C_{1}$ to the right server. For the same reason, the algorithm cannot move $C$ to the left server either. Thus, the algorithm’s only option to bring $C_{1}$ and $C$ to the same server is to replace $C$ with some $C_{i}$ at the cost of $2\alpha(K+1)$.

We will refer to the merged connected component of size $2(K+1)$ as $D$. Note that $D$ must be on the left server. Now let $C_{i}$ be the connected component of size $K+1$ on the right server. The adversary adds an edge from a vertex in $D$ to a vertex in $C_{i}$. By the same reasoning as before, the algorithm must now pick some $C_{j}$, $j\neq i$, of size $K+1$ from the left server and swap it with $C_{i}$. This costs another $2\alpha(K+1)$.

The adversary continues the previous procedure until only a $C_{i}$ of size $K+1$ is left on the left server and then she connects $C_{i}$ and $C^{\prime}$. This gives the final partitioning of the vertices.

We observe that each vertex which is on the left server at the very end, has been on the right server exactly once during the execution of the algorithm. Thus, the costs paid by the algorithm must be $\Omega(\alpha n)$.

Note that OPT pays exactly $\alpha(K+1)$ because it can determine beforehand which $C_{i}$ must be moved to the right server and only move that connected component. Before, we have seen that any deterministic algorithm must pay at least $\Omega(\alpha n)$. Thus, the competitive ratio is $\Omega(n/K)$. ∎

Next, we prove the $\Omega(\log n)$ lower bound for the competitive ratio of deterministic algorithms.

Lemma 38**.**

Consider the setting with two servers which both have capacity $(1+\varepsilon)n/2$ for $\varepsilon\leq 0.98$.

Then for each deterministic online algorithm ON there exists an input sequence $\sigma$ such that the cost paid by ON is $\Omega(\alpha n\log n)$ and the cost paid by OPT is $O(\alpha n)$. Thus, the competitive ratio of every deterministic online algorithm is $\Omega(\log n)$.

Proof.

Choose an arbitrary initial assignment of $n$ vertices to the $\ell$ servers. Since we want to prove a lower bound, we can assume that $n$ is a power of 2. Thus, suppose that $n=2^{a}$ for $a\geq 1000$.

In our hard instance, we are creating a sequence of edge insertions which proceeds in $\Theta(\log n)$ rounds. When round $i$ starts, all connected components have size $2^{i}$ induced by the previously provided edges, and when round $i$ finishes, all connected components have size $2^{i+1}$. We show that ON pays $\Omega(\alpha n)$ in each round. This implies the claimed cost of $\Omega(\alpha n\log n)$ for ON . The cost for OPT follows immediately from Lemma 3 which states that OPT never pays more than $O(n)$ when there are only two servers.

When ON starts and no edge was provided by the adversary, all connected components have size $1=2^{0}$, i.e., the connected components are isolated vertices.

Now suppose round $i=0,\dots,\log n$ starts. By induction, all connected components have size $2^{i}$. We now define a sequence of edge insertions for round $i$ which forces ON to pay $\Omega(\alpha n)$ and after which all connected components have size $2^{i+1}$.

Let $z$ denote the current number of connected components of size $2^{i}$. When round $i$ starts, there are exactly $z=n/2^{i}=2^{a-i}$ connected components of size $2^{i}$ each. Recall that each server has capacity $(1+\varepsilon)n/2$. Thus, at most

[TABLE]

connected components of size $2^{i}$ can be assigned to each server.

Now suppose there exists an edge $(u,v)$ such that $C_{u}$ and $C_{v}$ are of size $2^{i}$ and they are assigned to different servers; we call such an edge expensive. When the adversary inserts an expensive edge, ON must pay $\Omega(\alpha 2^{i})$ for moving $C_{u}$ or $C_{v}$ to a different server.

The strategy of the adversary is to insert expensive edges as long as they exist. Once no expensive edges exist anymore, the adversary connects all remaining components of size $2^{i}$ arbitrarily until all components have size $2^{i+1}$.

Note that expensive edges exist as long as $z>y_{i}$ (because when this inequality is satisfied, not all connected components of size $2^{i}$ can be assigned to the same server). Furthermore, observe that when the adversary inserts an expensive edge, $z$ decreases by 2.

Now we prove a lower bound on the number of expensive edges $p$. By the previous arguments, $p$ must be large enough such that:

[TABLE]

Solving this inequality for $p$, we obtain

[TABLE]

We conclude that that adversary can perform $\Omega(2^{a-i})$ expensive edge insertions. Since for each of these edge insertions, ON must pay $\Omega(\alpha 2^{i})$, we obtain that the cost paid by ON in round $i$ is

[TABLE]

7 Sample Applications: A Distributed Union Find Algorithm and Online

$k$-Way Partitioning

In this section we provide two sample applications for our model and our algorithms. First, we show that our results can be used to solve a distributed union find problem and we give an example where a union find data structure is used in practice. Second, we show that our algorithms imply competitive algorithms for an online version of the $k$-way partitioning problem.

7.1 Distributed Union Find

Recall that in the static union find problem, there are $n$ elements from a universe $\mathcal{U}$ and initially there are $n$ sets containing one element each. The data structure supports two operations: $\operatorname{union}(u,v)$ and $\operatorname{find}(u)$. Given two elements $u,v\in\mathcal{U}$, the operation $\operatorname{union}(u,v)$ merges the sets containing $u$ and $v$. The operation $\operatorname{find}(u)$ returns the set containing $u$.

In the distributed setting we consider, elements are stored across $\ell$ servers. Each server has enough capacity to store $(1+\varepsilon)n/\ell$ elements and we have the natural constraint that elements from the same set must always be stored on the same server (in order to maintain locality for elements from the same set). We consider a setting in which all sets have size $n/\ell$ when the algorithm finishes.

Note that if the sets of $u,v\in\mathcal{U}$ are stored on different servers when the operation $\operatorname{union}(u,v)$ is performed, one of the sets containing $u$ or $v$ must be moved to a different server. The goal of an algorithm is to minimize the moving cost caused by $\operatorname{union}$-operations.

When analyzing the moving cost, we will compare with an optimal offline algorithm which knows in advance which $\operatorname{union}$-operations will be performed. Thus, the optimal algorithm can move from the initial assignment to the final assignment at the minimum possible cost. For our analysis we will compute the competitive ratio between an online algorithm solving the above problem and the optimal offline algorithm (as also detailed in Section 2).

Using the algorithms from Sections 4 and 5.1, we obtain the following result.

Theorem 39**.**

Consider a system with $\ell$ servers each of capacity $(1+\varepsilon)n/\ell$ for $\varepsilon\in(0,1/2)$. Then there exists a distributed $O((\ell\log n\log\ell)/\varepsilon)$-competitive algorithm for the distributed union find problem. Moreover, for $\ell=O(\sqrt{\varepsilon n})$ servers, the algorithm’s communication cost does not exceed its cost for moving vertices.

Proof.

The theorem follows immediately from Theorem 29 by the following reduction from the model in Section 2. We identify vertices in the model from Section 2 with elements from the universe $\mathcal{U}$ in the union find model. Furthermore, for each operation $\operatorname{union}(u,v)$ we insert an edge $(u,v)$ into the model from Section 2. Since all algorithms we considered always collocate vertices from the same connected component, they satisfy the constraint that elements from the same set must be assigned to the same server. Moreover, in our analysis we were able to focus on the number of vertex moves due to Lemma 1. In our proofs, we showed competitive bounds for the number of vertex moves performed by the algorithm from Theorem 29 compared with an optimal offline algorithm. Thus, the same bounds as derived in Theorem 29 apply. ∎

For $\ell=\Omega(\sqrt{\varepsilon n})$ servers and the exact number of messages sent by the algorithm, see Theorem 29. The guarantees from Theorem 29 carry over immediately.

An examples where distributed union find data structures are used in practice is search engines [17]. A search engine stores many different documents from the Web over multiple servers. Now union find data structures are used to collocate duplicate documents on the same server, i.e., when documents $u$ and $v$ are identified as duplicates the operation $\operatorname{union}(u,v)$ is used to collocate these documents (and all previously identified duplicates) on the same server. Furthermore, union find data structures are used to find blocks in dense linear systems and in pattern recognition tasks (see Cybenko et al. [19] and references therein).

7.2 Online $k$-Way Partitioning

The model and algorithms we study in this paper can also be used to solve an online variant of the $k$-way partition problem [48]. In the static version of the $k$-way partition problem one is given a (multi-)set of integers $\mathcal{S}$ and the task is to partition $\mathcal{S}$ into $k$ subsets $\mathcal{S}_{1},\dots,\mathcal{S}_{k}$ such that the sum of all subsets is (approximately) equal.

Our model and our algorithms can be used to solve the following online version of this fundamental problem. Initially, $\mathcal{S}$ contains $n$ integers and all integers are $1$. Each integer is assigned to one of $\ell$ bins and each bin has capacity $(1+\varepsilon)n/\ell$. Now in an online sequence of operations, an adversary picks two integers from $\mathcal{S}$ and these integers are added. For example, after adding integers $a,b\in\mathcal{S}$, $\mathcal{S}$ becomes $\mathcal{S}=(\mathcal{S}\cup\{a+b\})\setminus\{a,b\}$. During this sequence of operations an online algorithm must ensure that the load of all bins is always bounded by $(1+\varepsilon)n/\ell$. We work under the assumption that after each operation there always exists an assignment from the integers in $\mathcal{S}$ to the bins such that each bin has load exactly $n/\ell$. We further assume that at the end of the sequence of operations there are $\ell$ integers and each integer is $n/\ell$.

Note that when two integers $a,b\in\mathcal{S}$ from different bins are added, either $a$ or $b$ must be moved to a different bin. This might cause that bin to exceed its capacity.

We will analyze algorithms which have small moving cost. That is, the cost of an algorithm is the sum of the numbers it has moved. We consider the competitive analysis of online algorithms compared with an optimal offline algorithm which knows the sequence of additions in advance and which can move the numbers at optimal cost.

We then obtain the following result for the $k$-way partitioning problem.

Theorem 40**.**

Consider a system with $\ell$ bins each of capacity $(1+\varepsilon)n/\ell$ for $\varepsilon\in(0,1/2)$. Then there exists a $O((\ell\log n\log\ell)/\varepsilon)$-competitive algorithm for the $k$-way partition problem.

Proof.

We can relate the online version of the $k$-way partition problem to the model we study by identifying integers and the sizes of connected components. Initially, we identify each $s\in\mathcal{S}$ with a single vertex. Note that this can be done since initially $s=1$ and thus $s$ and the size of its corresponding connected component are the same. After that, when two integers $a$ and $b$ are added, we take their corresponding connected components $C_{a}$ and $C_{b}$ and insert an edge between them. Note that the resulting integer $a+b$ corresponds to the connected component $C_{a}\cup C_{b}$ and their sizes agree, i.e., $a+b=|C_{a}\cup C_{b}|$. Now observe that summing the moving cost for integers is the same as counting the number of vertex reassignments for connected components. Thus, the result of the theorem follows from Theorem 16. ∎

8 Related Work

The design of more flexible networked systems that can adapt to their workloads has received much attention over the last years, with applications for traffic engineering [32, 33], load-balancing [43, 20], network slicing [49], server migration [12], switching [16, 25], or even adjusting the network topology [29]. The impact of distributed applications on the communication network is also well-documented in the literature [40, 26, 37, 50, 18]. Several empirical studies exploring the spatial and temporal locality in traffic patterns found evidence that these workloads are often sparse and skewed [4, 29, 47, 34], introducing optimization opportunities. E.g., studies of reconfigurable datacenter networks [30, 29] have shown that for certain workloads, a demand-aware datacenter network can achieve a performance similar to a demand-oblivious datacenter network at 25-40% lower cost [30, 29].

However, much less is known about the algorithmic challenges underlying such workload-adaptive networked systems, the focus of our paper. From an online algorithm perspective, our problem is related to reconfiguration problems such as online page (resp. file) migration [10, 14] as well as server migration [13] problems, $k$-server [24] problems, or online metrical task systems [15]. In contrast to these problems, in our model, requests do not appear somewhere in a graph or metric space but between communication partners. From this perspective, our problem can also be seen as a “distributed” version of online paging problems [23, 39, 51, 54] (and especially their variants with bypassing [2, 21]) where access costs can be avoided by moving items to a cache: in our model, access costs are avoided by collocating communication partners on the same server (a “distributed cache”).

The static version of our problem, how to partition a graph, is a most fundamental and well-explored problem in computer science [53], with many applications, e.g., in community detection [1]. The balanced graph partitioning problem is related to minimum bisection problems [22], and known to be hard even to approximate [6]. The best approximation today is due to Krauthgamer [36]. In contrast, we in this paper are interested in a dynamic version of the problem where the edges of the to-be-partitioned graph are revealed over time, in an online manner. Further, the offline problem of embedding workloads in a communication-efficient manner has been studied in the context of the minimum linear arrangement problem [46] and the virtual network embedding problem [55], however, without considering the option of migrations. In this regard, our paper features an interesting connection to the itinerant list update model [42], a kind of “dynamic” minimum linear arrangement problem which allows for reconfigurations and, notably, considers pair-wise requests. However, communication is limited to a linear line and so far, only non-trivial offline solutions are known.

One of the applications of the problem we study is a distributed union find data structure (see Section 7.1). Union find data structures have been initially proposed in the centralized setting and efficient algorithms were derived [27, 52]. Later, parallel versions of union find data structures were considered in a shared memory setting in which the goal was to derive wait-free algorithms [5]; also external memory algorithms were considered [3]. To the best of our knowledge studies of union find data structures in a distributed memory setting were only conducted experimentally, see (for example) [19, 38, 44, 45].

The second application we presented was as online $k$-way partitioning (Section 7.2). The $k$-way partitioning problem is known to be NP-hard as it constitutes a very simple scheduling problem [28]. The problem has also been researched in practice, see, e.g., [35, 48] and references therein. We are not aware of literature studying the online version of the problem which we have considered.

The paper most closely related to ours is by Avin et al. [8, 7] who studied a more general version of the problem considered in our paper. In their model, request patterns can change arbitrarily over time, and in particular, do not have to follow a partition and hence “cannot be learned”. Indeed, as we have shown in this paper, learning algorithms can perform significantly better: in [8], it was shown that for constant $\ell$ any deterministic online algorithm must have a competitive ratio of at least $\Omega(n)$ unless it can collocate all nodes on a single server, while we have presented an $O(\log{n})$-competitive online algorithm. Thus, our result is exponentially better than what can possibly be achieved in the model of [8].

9 Conclusion

Motivated by the increasing resource allocation flexibilities available in modern compute infrastructures, we initiated the study of online algorithms for adjusting the embedding of workloads according to the specific communication patterns, to reduce communication and moving costs. In particular, we presented algorithms and derived upper and lower bounds on their competitive ratio.

We believe that our work opens several interesting questions for future research. In particular, it remains to close the gap between the upper and lower bound of the competitive ratios derived in this paper. Furthermore, while in this paper we assumed that there are $\ell$ ground truth components of size $n/\ell$, it will be interesting to study more general settings with smaller and larger components.

More generally, it will be interesting to consider algorithms which do not collocate all communication partners. Also, studying collocation in specific networks such as Clos networks, which are frequently encountered in datacenters, would be intriguing.

Acknowledgments

We are grateful to our shepherd Rachit Agarwal as well as the anonymous reviewers whose insightful comments helped us improve the presentation of the paper.

The research leading to these results has received funding from the European Research Council under the European Community’s Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement No. 340506. Stefan Neumann gratefully acknowledges the financial support from the Doctoral Programme “Vienna Graduate School on Computational Optimization” which is funded by the Austrian Science Fund (FWF, project no. W1260-N35).