Distributed dynamic load balancing for task parallel programming

TL;DR

This paper presents a dynamic load balancing method for distributed task parallel applications using task migration and random search for process pairing, demonstrating a 5% reduction in execution time in specific tests.

Contribution

It introduces a novel load balancing strategy based on task migration and random process pairing for distributed task parallel software.

Findings

Achieved approximately 5% reduction in execution time.

Effective process pairing through random search within few steps.

Validated approach on block Cholesky factorization implementation.

Abstract

In this paper, we derive and investigate approaches to dynamically load balance a distributed task parallel application software. The load balancing strategy is based on task migration. Busy processes export parts of their ready task queue to idle processes. Idle--busy pairs of processes find each other through a random search process that succeeds within a few steps with high probability. We evaluate the load balancing approach for a block Cholesky factorization implementation and observe a reduction in execution time on the order of 5\% in the selected test cases.