HDArray: Parallel Array Interface for Distributed Heterogeneous Devices

TL;DR

HDArray offers a unified parallel array interface that simplifies programming across distributed heterogeneous devices by automating data and work distribution, reducing complexity in hybrid MPI and OpenCL environments.

Contribution

It introduces a novel array programming interface that automates data and work distribution across heterogeneous clusters, streamlining development.

Findings

Automatic communication among processes and devices

Simplified programming model for heterogeneous clusters

Enhanced productivity in parallel array programming

Abstract

Heterogeneous clusters with nodes containing one or more accelerators, such as GPUs, have become common. While MPI provides inter-address space communication, and OpenCL provides a process with access to heterogeneous computational resources, programmers are forced to write hybrid programs that manage the interaction of both of these systems. This paper describes an array programming interface that provides users with automatic and manual distributions of data and work. Using work distribution and kernel def and use information, communication among processes and devices in a process is performed automatically. By providing a unified programming model to the user, program development is simplified.