Experimental Analysis of Algorithms for Coflow Scheduling

TL;DR

This paper evaluates various algorithms for coflow scheduling in data centers, demonstrating that simple algorithms perform near-optimally and are robust across different settings, based on experiments with real and simulated data.

Contribution

It provides a comprehensive experimental comparison of coflow scheduling algorithms, including previously developed approximation algorithms, highlighting their practical effectiveness.

Findings

Simple algorithms are effective approximations of the optimal.

Approximation algorithms show robust and near-optimal performance.

Performance remains strong in both offline and online scenarios.

Abstract

Modern data centers face new scheduling challenges in optimizing job-level performance objectives, where a significant challenge is the scheduling of highly parallel data flows with a common performance goal (e.g., the shuffle operations in MapReduce applications). Chowdhury and Stoica introduced the coflow abstraction to capture these parallel communication patterns, and Chowdhury et al. proposed effective heuristics to schedule coflows efficiently. In our previous paper, we considered the strongly NP-hard problem of minimizing the total weighted completion time of coflows with release dates, and developed the first polynomial-time scheduling algorithms with O(1)-approximation ratios. In this paper, we carry out a comprehensive experimental analysis on a Facebook trace and extensive simulated instances to evaluate the practical performance of several algorithms for coflow scheduling, including the approximation algorithms developed in our previous paper. Our experiments suggest that simple algorithms provide effective approximations of the optimal, and that the performance of our approximation algorithms is relatively robust, near optimal, and always among the best compared with the other algorithms, in both the offline and online settings.