Methods for Partitioning Data to Improve Parallel Execution Time for Sorting on Heterogeneous Clusters

TL;DR

This paper presents new methods for partitioning data to optimize parallel sorting on heterogeneous clusters, reducing execution time through techniques tailored for different processor speed relations.

Contribution

It introduces efficient partitioning techniques for heterogeneous processors, including constant-time methods for related processors and dynamic programming for non-uniform cases.

Findings

Constant time partitioning for related processors

Dynamic programming approach for non-uniform processors

Solutions are generally in O(p) complexity, independent of problem size

Abstract

The aim of the paper is to introduce general techniques in order to optimize the parallel execution time of sorting on a distributed architectures with processors of various speeds. Such an application requires a partitioning step. For uniformly related processors (processors speeds are related by a constant factor), we develop a constant time technique for mastering processor load and execution time in an heterogeneous environment and also a technique to deal with unknown cost functions. For non uniformly related processors, we use a technique based on dynamic programming. Most of the time, the solutions are in O(p) (p is the number of processors), independent of the problem size n. Consequently, there is a small overhead regarding the problem we deal with but it is inherently limited by the knowing of time complexity of the portion of code following the partitioning.