Near Optimal Coflow Scheduling in Networks

TL;DR

This paper introduces a practical, efficient approximation algorithm for coflow scheduling in general networks, extending prior work from bipartite graphs to more complex graph models, with strong theoretical and empirical results.

Contribution

It presents a randomized 2-approximation algorithm for coflow scheduling on general graphs, improving previous bounds and demonstrating practical effectiveness.

Findings

The algorithm achieves a 2-approximation ratio.

Extensive experiments show the algorithm is practical and performs well.

The approach extends coflow scheduling to general network topologies.

Abstract

The coflow scheduling problem has emerged as a popular abstraction in the last few years to study data communication problems within a data center. In this basic framework, each coflow has a set of communication demands and the goal is to schedule many coflows in a manner that minimizes the total weighted completion time. A coflow is said to complete when all its communication needs are met. This problem has been extremely well studied for the case of complete bipartite graphs that model a data center with full bisection bandwidth and several approximation algorithms and effective heuristics have been proposed recently. In this work, we study a slightly different model of coflow scheduling in general graphs (to capture traffic between data centers) and develop practical and efficient approximation algorithms for it. Our main result is a randomized 2 approximation algorithm for the single path and free path model, significantly improving prior work. In addition, we demonstrate via extensive experiments that the algorithm is practical, easy to implement and performs well in practice.

Equations

$$\sum _j{w}_j{C}_j$$

$$\sum _t{x}_j^i(t)=1$$

$$X_j(t)\le \sum _{\ell =1}^t{x}_j^i(\ell )$$

$$C_j\ge 1+\sum _t(1-X_j(t))$$

$$r_j^i\ge t\Rightarrow x_j^i(t)=0$$

$$x_j^i(t)\ge 0$$

$$C_j$$

$$=\sum _{\tau =1}^T\sum _{t\ge \tau }^T{x}_j(t)=\sum _{\tau =1}^T\left(\sum _{t=1}^T{x}_j(t)-\sum _{t=1}^{\tau -1}{x}_j(t)\right)$$

$$=\sum _{\tau =1}^T(1-X_j(\tau -1))=\sum _{\tau =0}^{T-1}(1-X_j(\tau ))$$

$$=1+\sum _{\tau =1}^{T-1}(1-X_j(\tau ))$$

$$\sum _{p_j^i\ni e}{x}_j^i(t)\cdot {\sigma }_j^i$$

$$\sum _{e\in {\delta }_{out}(s_j^i)}{x}_j^i(t,e)=x_j^i(t),$$

$$\sum _{e\in {\delta }_{in}(t_j^i)}{x}_j^i(t,e)=x_j^i(t),$$

$$\sum _{e\in {\delta }_{in}(v)}{x}_j^i(t,e)=\sum _{e\in {\delta }_{out}(v)}{x}_j^i(t,e),$$

$$\forall v\in V\backslash \{s_j^i,t_j^i\}$$

$$\sum _{j\in [n],i\in [n_j]}{x}_j^i(t,e)\cdot {\sigma }_j^i\le c(e),$$

$$\sum _j{w}_j{C}_j^{\ast }$$

$$C_j(\text{LP Sched})=\underset{i}{\max }\{\underset{t:x_j^i(t)>0}{\max }[t]\}.$$

$$X_j(\tau )=X_j(\lfloor \tau \rfloor )+(\tau -\lfloor \tau \rfloor )\left(X_j(\lfloor \tau \rfloor +1)-X_j(\lfloor \tau \rfloor )\right).$$

$${\int }_{\tau =0}^T(1-$$

$$=T-\sum _{t=0}^{T-1}{\int }_{\tau =t}^{t+1}{X}_j(\tau )d\tau$$

$$=T-\sum _{t=0}^{T-1}{\int }_{\tau =t}^{t+1}\left[X_j(t)+(\tau -t)\left(X_j(t+1)-X_j(t)\right)\right]d\tau$$

$$=T-\sum _{t=0}^{T-1}\left[X_j(t)+\left(X_j(t+1)-X_j(t)\right){\int }_{\tau =t}^{t+1}(\tau -t)d\tau \right]$$

$$=T-\sum _{t=0}^{T-1}\frac{1}{2}\left[X_j(t)+X_j(t+1)\right]$$

$$=T-\left[\frac{1}{2}\left(X_j(0)+X_j(T)\right)+\sum _{t=1}^{T-1}{X}_j(t)\right]$$

$$=T-\left[\frac{1}{2}+\sum _{t=1}^{T-1}{X}_j(t)\right]$$

$$=1+\sum _{t=1}^{T-1}(1-X_j(t))-\frac{1}{2}$$

$$\le C_j^{\ast }-\frac{1}{2}$$

$${\int }_{\lambda =0}^1{C}_j^{\ast }(\lambda )\mathbf{d}\lambda$$

$$={\int }_{\tau =0}^T{\int }_{\lambda =0}^1\text{mathbbm}{1}_{[C_j^{\ast }(\lambda )>\tau ]}\mathbf{d}\lambda \mathbf{d}\tau$$

$$={\int }_{\tau =0}^T{\int }_{\lambda =X_j(\tau )}^{1}1\mathbf{d}\lambda \mathbf{d}\tau$$

$$={\int }_{\tau =0}^T(1-X_j(\tau ))\mathbf{d}\tau$$

$$\mathbb{E}[C_j(alg)]$$

$$\le {\int }_{\lambda =0}^1\left(2\lambda \right)\left(\frac{C_j^{\ast }(\lambda )}{\lambda }+1\right)d\lambda$$

$$=2{\int }_{\lambda =0}^1{C}_j^{\ast }(\lambda )d\lambda +1$$

$$=2{\int }_{\tau =0}^T(1-X_j(\tau ))\mathbf{d}\tau +1$$

$$\le 2\left(C_j^{\ast }-\frac{1}{2}\right)+1=2C_j^{\ast }$$

$$\sum _{j\in \{J\}}{w}_j\cdot {\mathbb{C}}_j$$

$$\le \sum _{j\in \{J\}}{w}_j\cdot \underset{i\in \{M\}}{\max }{C}_j^i$$

$$=\sum _{j\in \{J\}}{w}_j\cdot C_j$$

$$\text{Minimize}\sum _j{w}_j{C}_j$$

$$\sum _t{x}_j^i(t)=1$$

$$X_j(t)\le \sum _{\ell =1}^t{x}_j^i(\ell )$$

$$C_j\ge 1+\sum _t({\tau }_t-{\tau }_{t-1})(1-X_j(t))$$

$$r_j^i\ge {\tau }_t\Rightarrow x_j^i(t)=0$$

$$x_j^i(t)\ge 0$$

$$\sum _{p_j^i\ni e}{x}_j^i(t)\cdot {\sigma }_j^i$$

Full text

In this work, we study a slightly different model of coflow scheduling in general graphs (to capture traffic between data centers  (Jahanjou et al., 2017; You and Chowdhury, 2019)) and develop practical and efficient approximation algorithms for it. Our main result is a randomized 2 approximation algorithm for the single path and free path model, significantly improving prior work. In addition, we demonstrate via extensive experiments that the algorithm is practical, easy to implement and performs well in practice.

coflow, scheduling, LP relaxation, network flow, LP rounding, cloud computing

††journalyear: 2019††copyright: rightsretained††conference: 31st ACM Symposium on Parallelism in Algorithms and Architectures; June 22–24, 2019; Phoenix, AZ, USA††booktitle: 31st ACM Symposium on Parallelism in Algorithms and Architectures (SPAA ’19), June 22–24, 2019, Phoenix, AZ, USA††doi: 10.1145/3323165.3323179††isbn: 978-1-4503-6184-2/19/06††submissionid: spaaf023††ccs: Mathematics of computing Approximation algorithms††ccs: Mathematics of computing Graph algorithms††ccs: Networks Packet scheduling††ccs: Networks Network control algorithms††ccs: Theory of computation Scheduling algorithms††ccs: Theory of computation Routing and network design problems††ccs: Theory of computation Linear programming††ccs: Theory of computation Network flows††ccs: Theory of computation Rounding techniques††ccs: Computer systems organization Cloud computing

1. Introduction

Modern computing applications have rather intensive computational needs. Many machine learning applications require up to tens of thousands of machines and often involve processing units across multiple data centers collaborating on the same application. This collaboration is usually handled by a large-scale distributed computing framework that ideally ensures a close-to-linear speedup compared to a single machine. A crucial part of the collaboration is that large chunks of data require both inter and intra-datacenter transmissions.

For intra-datacenter transmission, a common example would be the MapReduce framework. Map workers write all intermediate results independently to several servers to guard against failure and allow possible re-calculation. These results are shuffled and sent to Reduce workers. The volume of transmission between machines is so large that it has become a major bottleneck in the performance. In addition to this challenge, multiple applications may share the same cluster, and an un-coordinated schedule of their data transmission may cause an unacceptable delay in their completion times.

Chowdhury and Stoica (Chowdhury and Stoica, 2012) first introduced the abstraction of coflow scheduling, which assumes that each application consists of a set of flows, and is finished once all the flows are completed. In their framework the network between machines is modeled as a switch: the input ports of different machines on one side, and output ports on the other side. A machine can send (receive) data to (from) any other machine, but to (from) only one machine at a time (sending and receiving may happen concurrently). The transmission speed between all machines is uniform. This describes a “perfect” datacenter where networking between machines is handled by a high-speed central switch (modeled by a complete bipartite graph) connected directly to all the machines (Chowdhury and Stoica, 2012). However, real world datacenters are far more complicated; direct (virtual) links between machines may exist to avoid latency, duplicate links may exist to tolerate failure, network speeds may vary widely for different machines and links, and complicated network structures may exist for a variety of reasons. To make things worse, some tasks may involve multiple datacenters around the globe, and the switch model simply cannot accurately capture the graph based network that connects all the data centers.

For inter-datacenter transmission, distributed machine learning tasks can generate huge amounts of traffic. Due to legal or cost reasons, some datasets cannot be gathered into a single datacenter for processing. Instead, several geographically distributed datacenters work together to train a single model, and exchange local updates frequently to ensure accuracy and convergence. Though the size of a single transmission may be small considering the network bandwidth, the repeated exchange blows up the volume of transmission and makes network traffic its bottleneck.

In order to solve these problems, a slightly different model of coflow scheduling was proposed by Jahanjou et al. (Jahanjou et al., 2017), which assumes that the underlying connection between machines is an arbitrary graph rather than a complete bipartite graph. Each node can be a machine, a datacenter or an exchange point (switch, router, etc.), and an edge between two nodes represents a physical link between the two Internet infrastructures. When some data needs to be transmitted from one node to another, it needs to be transmitted along edges. Unlike in the switch model where only one packet can be sent at each time slot, data for multiple jobs is allowed to transfer on the same link at the same time, or in other words, shared traffic on links is allowed. The total volume of data transmission on a link however is bounded by the link bandwidth111One major challenge in the switch model is the node-wise I/O speed constraint. In order to capture this in the graph model, we can replace every datacenter with a gadget of two nodes. The first node has exactly the same neighbors and edges that the original node for the datacenter has, plus links from and to the second node. The second node is only connected to the first node, and is the true source and destination for all demands involving this datacenter. By setting capacity on the links between these two nodes, we can enforce I/O limit for the whole datacenter like in the switch model.. Jahanjou et al. (Jahanjou et al., 2017) considered the model in which data has to travel along a single specified path. In addition to this model, we also consider the free path model which allows data to be split or merged at nodes to utilize the whole graph when transmitting the same piece of data as long as the capacity of each link is respected. This seems much more complicated in practice than a single path transmission, but modern distributed computation frameworks (You and Chowdhury, 2019) allow this kind of fine-grained control on network routing and transfer rate, which makes the model realistic. See Figure 1 for a brief illustration of the two models. The formal definitions come in Section 2.

1.1. Related Work

The idea of scheduling coflows was first introduced by Chowdhury and Stoica (Chowdhury and Stoica, 2012). Since then, it has been a hot topic in both the systems (Luo et al., 2016; Li et al., 2016; Chowdhury et al., 2014; Yu et al., 2016) and the theory (Qiu et al., 2015; Khuller and Purohit, 2016; Jahanjou et al., 2017; Ahmadi et al., 2017; Shafiee and Ghaderi, 2018) communities. Most theoretical research has focused on coflow scheduling in the switch model, where the communication graph is a complete, bipartite graph. Since this basic problem generalizes concurrent open shop scheduling and is thus NP-hard, the main results focus on the development of approximation algorithms. Over the last three years, a series of papers (Qiu et al., 2015; Khuller and Purohit, 2016; Ahmadi et al., 2017) have brought down the approximation factor from 67/3 to 5 for coflow scheduling with arbitrary release times and to 4 for the case without release times (Ahmadi et al., 2017; Shafiee and Ghaderi, 2018) 2224 is still the best known bound.. We would like to note that a very simple primal-dual framework is proposed by Ahmadi et al. (Ahmadi et al., 2017), and this yields a very practical combinatorial algorithm for the problem without requiring the need to solve an LP (as in (Shafiee and Ghaderi, 2018)). Furthermore, in recent work, a system called Sincronia (Agarwal et al., 2018) was also developed based on the primal-dual method. It improves upon state-of-the-art methods and gives practical and near-optimal solutions in real testbeds.

One natural extension is to take the graph structure into consideration. Zhao et al. (Zhao et al., 2015) consider coflow scheduling over arbitrary graphs and attempt to jointly optimize routing and scheduling. They give a heuristic based on shortest job first, and use the idle slots to schedule flows from the longest job. Jahanjou et al. (Jahanjou et al., 2017) studied two variants of coflow scheduling over general graphs, namely, when the path for a flow is given or if the path is unspecified. In both cases, the transmission rate may change over time, but each flow can only take a single path, whether given to or chosen by the fractional routing algorithm. In the first case, Jahanjou et al. (Jahanjou et al., 2017) develop the first constant approximation algorithm (approximation ratio $17.6$) and in the second case they develop an $O(\frac{\log n}{\log\log n})$ approximation algorithm ($n$ is the number of nodes in the graph), matching the lower bound given by Chuzhoy et al. (Chuzhoy et al., 2007).

Since preemption often incurs large overheads, some recent work (Yu et al., 2016) has tackled the problem of non-preemptive coflow scheduling. Mao, Aggarwal, and Chiang (Mao et al., 2018) consider the non-preemptive coflow scheduling problem with stochastic sizes and give an algorithm with an approximation factor of $(2\log m+1)(1+\sqrt{m}\Delta)(1+m\Delta)(3+\Delta)/2$, where $\Delta$ is an upper bound of squared coefficient of variation of processing times. This simplifies to a $(3\log m+\frac{3}{2})$ approximation for non-stochastic cases.

1.2. Our Contributions

The main result of this paper is a unified, tight $2$-approximation algorithm for the coflow scheduling problem in both the single path model and the free path model when all release times and demands are polynomially sized, and a $(2+\epsilon)$-approximation when the release times and demands can be super-polynomial. This improves upon the 17.6 approximation given by Jahanjou et al. (Jahanjou et al., 2017) for the single path model, and is the first approximation algorithm for the free path model (introduced by You and Chowdhury (You and Chowdhury, 2019)).

We also evaluated our algorithm using two WAN topologies (Microsoft’s SWAN (Hong et al., 2013) and Google’s G-Scale (Jain et al., 2013)) on four different workloads (BigBench (Intel Hadoop, 2016), TPC-DS (Nambiar and Poess, 2006), TPC-H (Poess and Floyd, 2000), and Facebook (FB) (Facebook, 2014; Chowdhury, 2015)) and compared with state-of-the-art for both models(Jahanjou et al., 2017; You and Chowdhury, 2019). For the single path model, we significantly improved over Jahanjou et al. (Jahanjou et al., 2017). For the free path model, we are close to what Terra (You and Chowdhury, 2019) gets, but have the extra capability of dealing with weights. Across all variants and models, we have shown that taking the LP solution directly is an effective heuristic in practice.

1.3. Paper Organization

In Section 2 we give a formal definition of the two models for coflow scheduling. In Section 3 we give a general linear program that deals with both models. We give the additional flow constraints for the two models in Section 3.1. In Section 4.1 we describe the main algorithm and present the analysis in Section 4.2. We prove both models to be NP-hard in Section 5. In Section 6, we show experimental results by comparing our algorithms to some baseline algorithms. We conclude in Section 7 with some new directions to work on.

2. Model and Problem Definition

We now formally define the models of coflow scheduling that we consider in this paper. Let $G=(V,E)$ be a directed graph that represents the data center network and $c:E\rightarrow\mathbb{R}^{+}$ be a function that denotes the capacity (bandwidth) available on each edge of the network. Let $\mathcal{J}=\{F_{1},F_{2},\ldots,F_{n}\}$ denote the set of $n$ coflows. A coflow $F_{j}$ has weight $w_{j}$ that denotes its priority and consists of $n_{j}$ individual flows, i.e., $F_{j}=\{f_{j}^{1},\ldots,f_{j}^{n_{j}}\}$ where $f_{j}^{i}=(s_{j}^{i},t_{j}^{i},\sigma_{j}^{i})$ denotes a flow from source node $s_{j}^{i}\in V$ to sink $t_{j}^{i}\in V$ with demand $\sigma_{j}^{i}\in\mathbb{R}^{+}$. We assume that time is discrete and data transfer is instantaneous, i.e., it takes negligible time for data to cover multiple hops of edges as network delay is low compared to the time to transmit large chunks of data. A coflow $F_{j}$ is said to be completed at the earliest time $t$ such that for each flow $f_{j}^{i}\in F_{j}$, $\sigma_{j}^{i}$ units of data have been transferred from source $s_{j}^{i}$ to sink $t_{j}^{i}$. Our goal is to find a schedule that routes all the requisite flows (i.e. at any time, what fraction of a certain flow is transmitted and along which path/paths) subject to the edge bandwidth constraints so that the total weighted completion time of the coflows $\sum_{j}w_{j}C_{j}$ is minimized. Figure 2 gives an example of an instance of the coflow scheduling problem over a simple network.

We consider two different transmission models, based on whether a flow $f_{j}^{i}$ has restrictions as to how the data is transmitted. In the single path model, each flow $f_{j}^{i}$ specifies a path $p_{j}^{i}$ from source $s_{j}^{i}\in V$ to sink $t_{j}^{i}\in V$ so that the flow can only be routed along that path. This is exactly the “circuit-based coflows with paths given” model studied by Jahanjou et al. (Jahanjou et al., 2017).

In the free path model, we can freely select the routing we desire for any flow $f_{j}^{i}$. In any time slot, data transmission occurs as a feasible multi-commodity flow so that both flow-conservation and edge bandwidth constraints are satisfied. Thus, we can split any flow $f_{j}^{i}$ along multiple paths from its source to destination. This model was proposed in Terra (You and Chowdhury, 2019). Since the shortest paths of different flows can share edges and cause congestion, the free path model offers the flexibility of rerouting flows along less congested paths. In addition, modern internet infrastructures support using multiple paths together to get a higher overall speed (known as link aggregation), which is captured in the free path model as network flow.

In fact, both models are handled uniformly by the same framework, and the only difference is the set of flow constraints that describe what are considered feasible transmissions. It is also possible to handle other kinds of transmissions, like an intermediate case between single path and free path: several paths are given, and we can use them together and decide at what rate we are transmitting along each path. Figures 3 and 4 show the optimal solutions for the example coflow problem in the single path and free path models respectively.

3. Linear Programming Relaxation

We use a time-indexed linear program to model this problem. Let $T$ denote an upper bound on the total time required to schedule all the coflows. Note that $T$ might be super-polynomial if the release times or coflow sizes are large. However, there is a standard technique that achieves polynomial size at the cost of a $(1+\epsilon)$ factor on approximation ratio. We will assume $T$ to be polynomial in the main paper, and present the fix for super-polynomial $T$ in Appendix A.

Let time be slotted and time slot $t$ cover the interval of time $[t-1,t]$. For a given flow $f^{i}_{j}$ and a time slot $t$, we introduce the variable $x_{j}^{i}(t)$ to indicate the fraction of flow $f^{i}_{j}$ that is scheduled at time $t$. For each coflow $F_{j}$, we introduce variables $X_{j}(t)$ to indicate if all the flows $f^{i}_{j}\in F_{j}$ have been completely scheduled by time $t$. Finally, we introduce a variable $C_{j}$ that models the completion time of coflow $F_{j}$.

To make the linear program compatible with both single path model and free path model, we exclude the flow constraints and edge bandwidth constraints for now and delay them to Section 3.1.

[TABLE]

Constraint (1) certifies that each flow is fully scheduled. Constraint (2) ensures that coflow $F_{j}$ is considered completed at time $t$ only if all flows $f_{j}^{i}\in F_{j}$ have been fully scheduled by time $t$. In Proposition 3.1, we show that Constraint (3) enforces a valid lower bound on the completion time of coflow $F_{j}$. Finally, Constraint (4) ensures that no flow is scheduled before it has been released. Note this is not a typical LP relaxation, since any fractional solution is valid. The main relaxation is around the completion time, since representing the exact completion time of job is beyond the capability of a linear program.

Proposition 3.1.

The completion time of a coflow $F_{j}$ can be lower bounded by $C_{j}\geq 1+\sum_{t}(1-X_{j}(t))$ where $X_{j}(t)\in[0,1]$ denotes the fraction of coflow $F_{j}$ that has been completed by (the end of) time slot $t$.

Proof.

Conventionally, in time-indexed linear programming relaxations, the completion time of a job $j$ is lower bounded by the fractional completion time in the schedule, or $C_{j}=C_{j}\cdot\sum_{t=1}^{T}x_{j}(t)\geq\sum_{t=1}^{T}t\cdot x_{j}(t)$. In our setting, this corresponds to the constraint $C_{j}\geq\sum_{t}t\cdot x_{j}(t)$ where $x_{j}(t)=X_{j}(t)-X_{j}(t-1)$ denotes the fraction of coflow $F_{j}$ that is scheduled during time slot $t$. The desired constraint in Eq (3) is exactly the same constraint rearranged in a format that is more convenient for analysis.

[TABLE]

∎

3.1. Model-specific Constraints

3.1.1. Single Path Model

In the single path model, a flow $f^{i}_{j}$ can only be routed along a specified path $p^{i}_{j}$. Thus, we do not need to make any routing decisions in the linear program and only need to ensure that edge bandwidths are respected.

[TABLE]

Constraint (6) enforces that the total flow scheduled through edge $e$ at any time slot $t$ does not exceed the edge bandwidth. Constraints (1)-(6) thus form the complete linear programming relaxation for coflow scheduling in the single path model.

3.1.2. Free Path Model

In the free path model, the path for flow $f^{i}_{j}$ is not specified. In fact, data can split and merge at vertices to utilize all possible capacity. We use variable $x^{i}_{j}(t,e)$ to denote the fraction of flow $f^{i}_{j}$ transmitted through edge $e$ in time slot $t$. Recall that we use $x^{i}_{j}(t)$ to denote the total fraction of flow $f^{i}_{j}$ that is transmitted in time slot $t$. $\delta_{in}(v)$ ($\delta_{out}$) represents the set of edges that comes in (out of) vertex $v$. Here are the flow conservation constraints we need.

[TABLE]

Constraints (7) and (8) enforce that the total fraction of flow $f^{i}_{j}$ satisfied at time $t$ over all the paths is exactly $x_{j}^{i}(t)$. Constraints (9) ensure flow conservation at all nodes other than source and sink. Constraints (10) guarantee that all edge bandwidths are satisfied at all time steps. Constraints (1)-(5) and (7)-(10) thus form the complete linear programming relaxation for coflow scheduling in the free path model.

Let $C_{j}^{*}$ denote the completion time of coflow $F_{j}$ in an optimal solution of the LP relaxation, and let $C_{j}(opt)$ denote the completion time of coflow $F_{j}$ in the corresponding optimal integral solution. Thus, for both the models, we have

[TABLE]

4. Approximation Algorithms

Let $x_{j}^{i}(t)$ denote the fraction of flow $f_{j}^{i}$ that is scheduled at time step $t$ in an optimal solution to the above LP. The LP constraints guarantee that this yields a feasible schedule to the coflow scheduling problem (in both the single path as well as the free path models). However, since the completion time of a coflow $F_{j}$ is defined as the earliest time $t$ such that all flows $f_{j}^{i}\in F_{j}$ have been completely scheduled, the true completion time of coflow $F_{j}$ obtained in this scheduled is given by

[TABLE]

Unfortunately, this completion time $C_{j}(\text{\it LP Sched})$ can be much greater than the completion time variable in the optimal LP solution $C_{j}^{*}$, and thus the obtained schedule is not a constant-approximate coflow schedule. For instance, consider a coflow $F_{j}$ with only one flow ($n_{j}=1$) and let the optimal LP solution set its schedule as follows $x^{1}_{j}(1)=0.9,x^{1}_{j}(10)=0.1$, and $x^{1}_{j}(t)=0,\forall t\notin\{1,10\}$. Now, the completion time variable in the optimal LP solution is $C_{j}^{*}=\sum_{t}tx_{j}^{1}(t)=1.9$. However, true completion time of the coflow $F_{j}$ in such a schedule is $C_{j}(\text{\it LP Sched})=10\gg C_{j}^{*}$.

To overcome the obstacle above, we propose the following algorithm called Stretch (see Section 4.1) that modifies the schedule obtained by the linear program so that the completion time of each coflow in the modified schedule can be compared with the completion time variable of the corresponding coflow in an optimal LP solution. The schedule “stretching” idea (also called ‘slow-motion’) used in our algorithm has been used before successfully in other scheduling contexts (Im et al., 2014; Queyranne and Sviridenko, 2002; Schulz and Skutella, 1997).

4.1. Stretch Algorithm

1. (1)

Solve the linear program in Section 3 and obtain a fractional optimal solution. 2. (2)

Let $\lambda\in(0,1)$ be drawn randomly according to the p.d.f $f(v)=2v$. We can verify that this is indeed a valid probability distribution. 3. (3)

Stretch the LP schedule by $\frac{1}{\lambda}$. This means that we schedule everything exactly as per the LP solution - but whatever LP schedules in the interval $[a,b]$, we will schedule in the interval $[\frac{a}{\lambda},\frac{b}{\lambda}]$. 4. (4)

Once $\sigma_{j}^{i}$ units of flow $f_{j}^{i}$ have been scheduled, leave the remaining slots for $f_{j}^{i}$ empty.

Figure 5 illustrates the key ideas of the algorithm. To help understand this algorithm, start with the simple case where we have a fixed $\lambda=0.5$, in other words stretch the time axis by a factor of $1/\lambda=2$. Intuitively, we move everything at time slot $t$ and to both time slots $2t-1$ and $2t$. What used to be transmitted at time $t$ will be transmitted no later than time $2t$. Consider any flow $f^{i}_{j}$ and let $\tau$ denote the earliest time by which the LP has scheduled at least $1/2$ fraction of the flow. Then, it is easy to verify that the flow $f^{i}_{j}$ is completely scheduled by time $2\tau$.

Now we consider a general $\lambda$ and prove that this algorithm does output a feasible schedule. Due to fractional $\lambda$, it might be the case that some flow $f_{j}^{i}$ of LP variable $x_{j}^{i}(t)$ in integral interval $[t-1,t]$ becomes $[\frac{t-1}{\lambda},\frac{t}{\lambda}]$, a fractional interval. In this case, for a time slot $\tau$, or a interval $[\tau-1,\tau]$ after stretching, we just add $x_{j}^{i}(t)\cdot|[\tau-1,\tau]\cap[\frac{t-1}{\lambda},\frac{t}{\lambda}]|$.

The only flows that might be scheduled in time slot $\tau$ are those scheduled in time slot $1+\lfloor\lambda(\tau-1)\rfloor$ and $1+\lfloor\lambda\tau\rfloor$ before stretching, or flows $f_{j}^{i}(1+\lfloor\lambda(\tau-1)\rfloor)$ and flows $f_{j}^{i}(1+\lfloor\lambda\tau\rfloor)$. (The two time slots might be the same. If so, feasibility is automatically met. Otherwise, we have $1+\lfloor\lambda(\tau-1)\rfloor+1=1+\lfloor\lambda\tau\rfloor$.) For all flows at time $1+\lfloor\lambda(\tau-1)\rfloor$ before stretching, the factor we multiplied with is $w_{1}=\left|[\tau-1,\tau]\cap[\frac{\lfloor\lambda(\tau-1)\rfloor}{\lambda},\frac{1+\lfloor\lambda(\tau-1)\rfloor}{\lambda}]\right|$. For all flows at time $1+\lfloor\lambda\tau\rfloor$ before stretching, the factor we use to multiply with is $w_{2}=\left|[\tau-1,\tau]\cap[\frac{\lfloor\lambda\tau\rfloor}{\lambda},\frac{1+\lfloor\lambda\tau\rfloor}{\lambda}]\right|$. Note $w_{1}+w_{2}=1$. In fact, the schedule at time $\tau$ can be viewed as a weighted average of the schedule at time $[\lfloor\lambda(\tau-1)\rfloor,1+\lfloor\lambda(\tau-1)\rfloor]$ and $[\lfloor\lambda\tau\rfloor,1+\lfloor\lambda\tau\rfloor]$ (if $\lambda(\tau-1)$ is a integer, then the schedule will be exactly what it used to be at time $\lambda\tau$), the first with weight $w_{1}$ and the second with weight $w_{2}$. The nature of network flow ensures that the weighted sum of two feasible flows is a feasible flow.

Another fact that needs proof is that every flow is finished. This is guaranteed since schedules are stretched, and we only leave the remaining slots empty for $f_{j}^{i}$ if $\sigma_{j}^{i}$ units of flow have been scheduled, or in other words, all the demand for this flow has been scheduled.

4.2. Analysis

Recall that $C_{j}^{*}$ denotes the completion time of coflow $F_{j}$ in the optimal LP solution. While we consider that time is slotted in the LP formulation and time slot $t$ covers the interval of time $[t-1,t]$, at this stage it is more convenient to work with continuous time rather than discrete time. For any continuous time $\tau\in[0,T]$, define $X_{j}(\tau)$ to be the fraction of coflow $F_{j}$ that has been scheduled in the LP solution by time $\tau$. We define $X_{j}(\tau)$ by assuming that the flow is scheduled at an uniform rate in every time slot. Formally, we have

[TABLE]

The LP constraints (3) guarantee that for any coflow $F_{j}$, we have $C_{j}^{*}\geq 1+\sum_{t}(1-X_{j}(t))$. We can now lower-bound the LP completion time by replacing the above summation by an integral.

Lemma 4.1.

$\int_{\tau=0}^{T}(1-X_{j}(\tau))d\tau\leq C_{j}^{*}-\frac{1}{2}$* where $X_{j}(\tau)$ is defined as per Eq. (13).*

Proof.

By definition of $X_{j}(\tau)$, we have the following.

[TABLE]

where the last inequality follows from Constraint (3). ∎

For any $\lambda\in[0,1]$, define $C_{j}^{*}(\lambda)$ to be the earliest time $\tau$ such that $\lambda$ fraction of the coflow $F_{j}$ has been scheduled in the LP solution, i.e., in other words its the smallest $\tau$ such that $X_{j}(\tau)=\lambda$. Note that by time $C_{j}^{*}(\lambda)$, $\lambda$ fraction of every flow $f^{i}_{j}\in F_{j}$ has been scheduled by the LP.

Proposition 4.2.

$\displaystyle\int_{\lambda=0}^{1}C^{*}_{j}(\lambda)\mathbf{d}\lambda=\int_{\tau=0}^{T}(1-X_{j}(\tau))\mathbf{d}\tau$**

Proof.

[TABLE]

∎

Finally, we are ready to bound the completion time of coflow $F_{j}$ in the stretched schedule (denoted as $C_{j}(alg)$). For any fixed $\lambda\in(0,1)$, since we stretch the schedule by a factor of $\frac{1}{\lambda}$, we have $C_{j}(alg)\leq\left\lceil\frac{C_{j}^{*}(\lambda)}{\lambda}\right\rceil$. Notice the ceiling function in the bound 333All flows $f_{j}^{i}\in F_{j}$ were completed by at least $\lambda$ fraction by time $C^{*}_{j}(\lambda)$. So in the stretched schedule, all those flows must be completed by time $\frac{C^{*}_{j}(\lambda)}{\lambda}$. The ceiling is necessary since $\frac{C^{*}_{j}(\lambda)}{\lambda}$ may be fractional (i.e. occur in the middle of a time slot). Since $\lambda$ is drawn randomly from a distribution, the following lemma bounds the expected completion time of coflow $F_{j}$ in the stretched schedule.

Lemma 4.3.

The expected completion time of any coflow $F_{j}$ in the stretched schedule is bounded by $2C_{j}^{*}$.

Proof.

[TABLE]

∎

Theorem 4.4 thus follows from the linearity of expectation.

Theorem 4.4.

There is a randomized 2-approximation algorithm for coflow scheduling in networks in both the single path and free path models when all release times and coflow sizes are polynomially sized.

For the case where the total time we need to schedule all coflows is super-polynomial, we use the standard trick of geometric series time intervals, and claim the following theorem. Proof comes in Appendix A.

Theorem 4.5.

For any $\epsilon>0$, there is a randomized $(2+\epsilon)$-approximation algorithm for coflow scheduling in networks in both the single path and the free path models (with possibly super polynomial release times and demands).

5. Hardness of Approximation

We claim the following theorem:

Theorem 5.1.

For the coflow scheduling problem, in both the single path and the free path model, it is NP-hard to obtain a $(2-\epsilon)$ approximation, for any $\epsilon>0$.

Proof.

We prove it by a reduction from concurrent open-shop problem (proved NP-hard to approximate within a factor better than $(2-\epsilon)$ (Bansal and Khot, 2010; Sachdeva and Saket, 2013)). The definition of concurrent open shop problem is as follows: there are $m$ machines and $n$ jobs, each job $j$ need to be processed on machine $i$ for $p_{j}^{i}$ time non-preemptively. We would like to minimize the total weighted completion time. Unlike the open shop problem, in the concurrent open shop problem a job can be processed on more than one machine at the same time.

Given a concurrent open-shop problem instance with $M$ machines, we construct an instance of the coflow scheduling problem as follows. For every machine $i$, we have two nodes $x_{i}$ and $y_{i}$, and an edge of unit bandwidth from $x_{i}$ to $y_{i}$. Notice the graph has $M$ different components, between each pair $(x_{i},y_{i})$, there is only one path from $x_{i}$ to $y_{i}$. Thus this construction works for both the single path model and the free path model. We will not distinguish the models in the following proof.

For a certain job $j$ with demands $\sigma_{j}^{i}$ in the concurrent open shop instance, we add a coflow $j$ with demand of $\sigma_{j}^{i}$ from $x_{i}$ to $y_{i}$. Weights are directly taken from the concurrent open shop problem instance. Suppose we get a solution for this coflow scheduling instance, we can get a solution of no larger cost for the concurrent open shop instance as follows. If we have a flow $f_{j}^{i}$ for job $j$ on edge $(x_{i},y_{i})$ of size $x_{j}^{i}(t)$ at time $t$, then we schedule a fraction of $x_{j}^{i}(t)$ for job $j$ on machine $i$ at time $t$. Suppose a flow $f_{j}^{i}$ is finished at time $C_{j}^{i}$ in the coflow scheduling problem, the corresponding concurrent open shop problem for job $j$ and machine $i$ is also finished at time $C_{j}^{i}$. Similarly, the finishing time $C_{j}$ of coflow $j$ and concurrent open shop job $j$ are the same. However, the solution we get is fractional, and might be preemptive (we might pause a job and resume it later).

Now we prove that we can modify this solution to get a non-preemptive integral solution without raising the total weighted completion time. For each machine $i$, consider all completion times $C_{j}^{i}$. Sort them in non-decreasing order $C_{l_{1}}^{i},C_{l_{2}}^{i},\dots,C_{l_{J}}^{i}$, and we can safely reschedule these demand in the order of $l_{1},l_{2},\dots,l_{j}$, and get new completion times $\mathbb{C}_{l_{1}}^{i},\dots,\mathbb{C}_{l_{j}}^{i}$ while not raising any completion time. We know all demand of job $l_{1}$ on machine $i$ has been finished by $C_{l_{1}}^{i}$, so $\mathbb{C}_{l_{1}}^{i}=d_{l_{1},i}\leq C_{l_{1}}^{i}$, similarly all demands of job $l_{1}$ and $l_{2}$ have been finished by $C_{l_{2}}^{i}$, and $\mathbb{C}_{l_{2}}^{i}=d_{l_{1},i}+d_{l_{2},i}\leq C_{l_{2}}^{i}$. We can continue and get $\mathbb{C}_{l_{j}}^{i}\leq C_{l_{j}}^{i},\forall j\in\{J\}$. Thus the total weighted completion time for this integral solution would be upper bounded by the cost for the coflow scheduling instance.

[TABLE]

For the other direction, for a certain solution of a concurrent open-shop problem, if task $i$ of job $j$ is scheduled from time $t_{1}$ to time $t_{2}$, we make the flow $f_{j}^{i}$ take up all bandwidth of edge $(x_{i},y_{i})$ from time $t_{1}$ to time $t_{2}$. Then flow $f_{j}^{i}$ is finished the same time when task $i$ of job $j$ is finished. Since every task $i$ is finished the same time before and after reduction, completion times and the objective weighted completion time stays the same for the coflow scheduling problem.

In conclusion, for a solution $SOL$ of concurrent open-shop problem with weighted completion time $W$, we can construct a solution $SOL_{\text{coflow}}$ for coflow scheduling problem of the same weighted completion time $W$. For a solution $SOL^{\prime}_{\text{coflow}}$ of coflow scheduling problem with weighted completion time $W^{\prime}$, we can construct a solution $SOL^{\prime}$ for the original concurrent open-shop problem, with cost at most $W^{\prime}$. Since concurrent open-shop problem is NP-hard to get a $(2-\epsilon)$ approximation, we know it is also NP-hard to approximate coflow scheduling problem to a factor of $(2-\epsilon)$, for both single path model and free path model. ∎

6. Experiments

We evaluated the Stretch Algorithm on 2 topologies and 4 benchmarks/industrial workloads. Experiments were run on a machine with dual Intel(R) Xeon(R) CPU E5-2430, and 64GB of RAM, and using Gurobi (Gurobi Optimization, LLC, 2018) as the LP solver. We first discuss the experimental set up and then in Section 6.2 discuss what evaluation we performed.

WAN topology: We consider the following graph topologies.

1. (1)

Swan (Hong et al., 2013): Microsoft’s inter-datacenter WAN with 5 datacenters and 7 inter-datacenter links. We calculate link bandwidth using the setup described by Hong et al.(Hong et al., 2013). 2. (2)

G-Scale (Jain et al., 2013): Google’s inter-datacenter WAN with 12 datacenters and 19 inter-datacenter links.

Workloads: We use the following mix of jobs from public benchmarks - TPC-DS (Nambiar and Poess, 2006), TPC-H (Poess and Floyd, 2000), and BigBench (Intel Hadoop, 2016) - and from Facebook (FB) production traces (Facebook, 2014; Chowdhury, 2015). We follow (You and Chowdhury, 2019) to set up the benchmarks: for a certain workload, jobs are randomly chosen and since they do not have a release time, we assign a release time similar to that in production traces. Each job lasts from a few minutes to dozens of minutes. Each benchmark experiment has 200 jobs. We randomly assign these jobs to nodes in the datacenter, and the demand will be between the corresponding nodes. Since weights are not available, we assign weights that are uniformly chosen from the interval between $1.0$ and $100.0$.

6.1. Implementation Details

In this subsection we discuss some details related to the implementation.

Time Index: There is a trade-off in selecting the size of a time slot. If the length of a time slot is shorter, we get more accurate answers, but need to solve a larger LP. On the contrary, if we make each time slot longer, the amount of computational resources need is greatly reduced, but the quality of the solution suffers. In all our experiments, we considered time slots of length $50$ seconds as this led to tractable LP relaxations.

Rounding: Algorithm Stretch is meant for easy theoretical analysis, and is not a sophisticated rounding method; we are not trying to schedule later flows in the slots that are idle. This can cause huge overhead in experiments. See Figure 5 for an illustration. In our implementation, we deal with this issue by moving the schedule of every time slot $t$ to an earlier idle slot $t^{\prime}$ if for all flows scheduled at $t$, its release time is before $t^{\prime}$.

To address the random sampling of $\lambda$, we sample $20$ times from the distribution mentioned in Section 4.1 to get the expected weighted completion time for Algorithm Stretch, and denote it with “Average $\lambda$”. We also measure the best solution obtained over these random choices (denoted by “Best $\lambda$”).

6.2. Baselines

LP-based Heuristic: In addition to algorithms with theoretically worst case guarantee, we also propose a heuristic that works well in practice. Recall in Section 4.1, we mentioned that the LP solution itself is a valid schedule. We can use this solution as a heuristic, for both the single path and free path models. Note the weighted completion time for this LP solution is NOT the same as the LP objective function, as explained in Section 4.1. This implies that the solution from the heuristic can be arbitrarily bad in the worst case. In practice, however, this proves to be a very effective algorithm that can be quite close to the lower bound we get from LP.

Jahanjou et al. (Single path model): Since path information is not available in the datasets, we randomly generate one for each flow. For a source sink pair $(s_{j}^{i},t_{j}^{i})$, we randomly select one of the shortest paths as the path for flow $f_{j}^{i}$. For this model, we compare our algorithm with the algorithm presented by Jahanjou et al. (Jahanjou et al., 2017). Here is a brief description of their approach. First write an LP using geometric time intervals, then schedule each job according to the interval its $\alpha$ point (the time when $\alpha$ fraction of this job is finished) belongs to. A common reason for geometric time intervals is to avoid having a super-polynomial time horizon (a practical reason is to make the LP smaller), and a time series of $\{(1+\epsilon)^{i}\}$ is chosen where $\epsilon$ is close to 0. The closer $\epsilon$ is to 0, the better the approximation ratio can be. However, in Jahanjou et al.’s algorithm, the rounding step has a dependency on $\epsilon$. To optimize the approximation ratio, $\epsilon$ is set to $0.5436$. Our algorithm, on the contrary, is time slot based, and can be turned into a geometric series of time intervals by losing a factor of $(1+\epsilon)$. In experiments, we include both the case of $\epsilon=0.2$ and the case of $\epsilon=0.5436$ for completeness.

Terra (Free path model): For the flow-based model, we are comparing to the offline algorithm in Terra (You and Chowdhury, 2019). This algorithm only works for the unweighted case. It calculates the time for each single coflow to finish individually, and then schedule with SRTF (shortest remaining time first). Instead of one large LP like all other algorithms compared here, this algorithm solves a large number of LPs, twice the number of coflow jobs. Terra can work with very fine grained time, to the order of milliseconds (and does not need time to be slotted). Since there is no previous work on weighted case, we compare the weighted case with the LP solution and our heuristic directly based on time indexed LP.

6.3. Experimental Results

Impact of $\lambda$: See Figure 6 and Figure 7. When $\lambda$ is $1.0$, we take the LP solution directly (this is exactly the LP-based heuristic). Across all experiments, this seems the best choice of $\lambda$. The best sampled $\lambda$ and the average case $\lambda$ are pretty close, indicating the performance does not change much across different $\lambda$.

Impact of $\epsilon$: To study the effect of the size of the time interval, we measure the LP objective and the schedule obtained by the LP-based heuristic as we vary $\epsilon$ in Figure 8. As $\epsilon$ increases, the size of the linear program will drop, making it faster to solve. On the other hand, the quality of solution drops, as we will not start a job until the whole current interval is after its release time, and will not consider a job finished until the interval its completion time belongs to ends. Thus a proper selection of $\epsilon$ may depend on the available computational resources for solving the LP.

Single Path Model: Figures 9 and 10 compare the performance of our algorithms with that of Jahanjou et al. (Jahanjou et al., 2017) on all the benchmarks and topologies. Across all the experiments, we observe that our algorithms perform significantly better.

Free Path Model: See Figure 11 and Figure 12 for comparisons with the algorithm in Terra(You and Chowdhury, 2019). Since Terra only handles uniform coflow weights, we set all weights to be unit for these experiments. Surprisingly, we observe that Terra performs slightly better than even the LP objective itself. This disparity arises as the LP relies on time slots of 50 seconds while Terra deals with time slots of much finer granularity. For the weighted case, we are not aware of previous work, and only compare to LP solution in Figures 6 and 7.

7. Conclusion

In this paper we developed an efficient approximation algorithm for the coflow scheduling problem in general graph topologies. This algorithm is shown to be practical and one that delivers extremely high quality solutions. The new insight was to write a time indexed LP formulation and to convert it using the idea of stretching the schedule.

The next major challenge is developing online methods for coflow shceduling to minimize weighted flow time. Prior work (Khuller et al., 2018) deals with the problem of minimizing weighted completion time by making use of offline approximation algorithms. However, the problem of minimizing weighted flow time is considerably more challenging. The technical difference is that flow time is defined as $C_{j}-r_{j}$ where $C_{j}$ is the completion time of a job, and $r_{j}$ the release time. Optimizing flow time non-preemptively even on a single machine (a different model) is a notoriously difficult problem with some recent progress (Batra et al., 2018; Feige et al., 2019).

Appendix A Sketch of generalization to super-polynomial time span

Geometric series time interval is defined as follows. For an $\epsilon>0$, let $\tau_{0}=0,\tau_{1}=1,\cdots,\tau_{k}=(1+\epsilon)^{k-1},\cdots$. We define the $k$-th interval as $l_{k}=[\tau_{k-1},\tau_{k}]$. Since $T$ is at most the sum of all processing time and all release time, we know the number of intervals $\mathbb{T}=1+\lceil\log_{1+\epsilon}T\rceil$ is polynomial.

We change the LP as follows. We abuse notation a bit and allow $\mathbb{T}$ to represent the set $\{1,2,\cdots,\mathbb{T}\}$ when there is no confusion. We replace all accurance of $T$ with $\mathbb{T}$ in Section 3, modify Equation (4) and Equation (3) to accommodate for release time, and get the following linear program.

[TABLE]

For the model specific part of linear program, we only need to change the capacity constraints: replace Equation (6) for single path model to get

[TABLE]

and Equation (10) for free path model to get

[TABLE]

Similar to Proposition 3.1, we prove Constraint (16) is a good lower bound.

Proposition A.1.

The completion time of a coflow $F_{j}$ can be lower bounded by $C_{j}\geq 1+\sum_{t}(\tau_{t}-\tau_{t-1})(1-X_{j}(t))$ where $X_{j}(t)\in[0,1]$ denotes the fraction of coflow $F_{j}$ that has been completed by (the end of) time interval $[\tau_{t-1},\tau_{t}]$.

Proof.

If a job completes in the interval $(\tau_{t-1},\tau_{t}]$, then its finishing time is at least $\tau_{t-1}+1$.

[TABLE]

∎

After getting a solution, we would schedule coflows into intervals instead of into time slots. Inside each time interval, we just schedule each flow at uniform speed, and break into actual time slots. Similar to Section 4.1, we can prove that this solution is feasible.

A.1. Analysis

Recall that $C_{j}^{*}$ denotes the completion time of the coflow $F_{j}$ in the optimal LP solution. For any continuous time $t\in[0,T]$, define $\widehat{X}_{j}(t)$ to be the fraction of coflow $F_{j}$ that has been scheduled in the LP solution by time $t$. Note $X_{j}(t)$ is for time interval $[\tau_{t-1},\tau_{t}]$, but $\widehat{X}_{j}(t)$ is for original time slots. Flows are scheduled at an uniform rate in every time interval. Use $\rho(t)$ to denote the smallest $\rho$ such that $t\in(\tau_{\rho-1},\tau_{\rho}]$, we have

[TABLE]

Similar to Lemma 4.1, we state and prove the following lemma.

Lemma A.2.

$\int_{t=0}^{T}(1-\widehat{X}_{j}(t))\mathbf{d}t\leq(1+\epsilon)C_{j}^{*}-\frac{1}{2}$* where $X_{j}(\rho)$ is defined as per Eq. (24).*

Proof.

From constraints (16), we have that

[TABLE]

∎

For any $\lambda\in[0,1]$, define $C_{j}^{*}(\lambda)$ to be the earliest time $\rho$ such that $\lambda$ fraction of the coflow $F_{j}$ has been scheduled in the LP solution, i.e., in other words its the smallest $t$ such that $\widehat{X}_{j}(t)=\lambda$. Note that by time $C_{j}^{*}(\lambda)$, $\lambda$ fraction of every flow $f^{i}_{j}\in F_{j}$ has been scheduled by the LP.

Proposition A.3.

$\displaystyle\int_{\lambda=0}^{1}C_{j}(\lambda)\mathbf{d}\lambda\leq\int_{t=0}^{T}(1-\widehat{X}_{j}(t))\mathbf{d}t$.

Proof.

[TABLE]

∎

Finally, we are ready to bound the completion time $C_{j}(alg)$ of coflow $F_{j}$ in the stretched schedule. For any fixed $\lambda\in(0,1)$, since we stretch the schedule by a factor of $\frac{1}{\lambda}$, it is easy to verify444All flows $f_{j}^{i}\in F_{j}$ were completed by at least $\lambda$ fraction by time $C_{j}(\lambda)$. So in the stretched schedule, all those flows must be completed by time $\left\lceil\frac{C_{j}(\lambda)}{\lambda}\right\rceil$. that $C_{j}(alg)\leq\left\lceil\frac{C_{j}(\lambda)}{\lambda}\right\rceil$. Since $\lambda$ is drawn randomly from a distribution, the following lemma bounds the expected completion time of coflow $F_{j}$ in the stretched schedule.

Lemma A.4.

The expected completion time of any coflow $F_{j}$ in the stretched schedule is bounded by $2(1+\epsilon)C_{j}^{*}$.

Proof.

[TABLE]

∎

Theorem 4.5 thus follows from the linearity of expectation.