Scheduling Coflows with Precedence Constraints for Minimizing the Total Weighted Completion Time in Identical Parallel Networks

TL;DR

This paper presents new approximation algorithms for scheduling coflows with precedence constraints in parallel networks, significantly improving the efficiency of minimizing total weighted completion times in data-parallel computing.

Contribution

It introduces novel approximation algorithms for coflow scheduling with precedence constraints, applicable to divisible and indivisible coflows, and extends to multi-stage job scheduling.

Findings

Achieves improved approximation ratios for divisible coflows.

Provides algorithms with near-optimal performance for indivisible coflows.

Extends solutions to multi-stage job scheduling with precedence constraints.

Abstract

Coflow is a recently proposed network abstraction for data-parallel computing applications. This paper considers scheduling coflows with precedence constraints in identical parallel networks, such as to minimize the total weighted completion time of coflows. The identical parallel network is an architecture based on multiple network cores running in parallel. In the divisible coflow scheduling problem, the proposed algorithm achieves $(6-\frac{2}{m})\mu$ and $(5-\frac{2}{m})\mu$ approximate ratios for arbitrary release time and zero release time, respectively, where $m$ is the number of network cores and $\mu$ is the coflow number of the longest path in the precedence graph. In the indivisible coflow scheduling problem, the proposed algorithm achieves $(4m+1)\mu$ and $4m\mu$ approximate ratios for arbitrary release time and zero release time, respectively. In the single network core scheduling problem, we propose a $5\mu$-approximation algorithm with arbitrary release times, and a $4\mu$-approximation without release time. Moreover, the proposed algorithm can be modified to solve the coflows of multi-stage jobs scheduling problem. In multi-stage jobs, coflow is transferred between servers to enable starting of next stage. This means that there are precedence constraints between coflows of job. Our result represents an improvement upon the previous best approximation ratio of $O(\tilde{\mu} \log(N)/ \log(\log(N)))$ where $\tilde{\mu}$ is the maximum number of coflows in a job and $N$ is the number of servers.