Scalable data abstractions for distributed parallel computations

TL;DR

This paper introduces a scalable programming model for distributed parallel computations that separates data representation from computation, enabling efficient, flexible, and scalable parallel programs on large systems.

Contribution

It proposes a simple, scalable programming model with distributed data abstractions that can be efficiently implemented and directly compiled to hardware.

Findings

Supports both shared and distributed data-parallelism styles

Capable of efficient hardware-level implementation

Enhances scalability of parallel programs

Abstract

The ability to express a program as a hierarchical composition of parts is an essential tool in managing the complexity of software and a key abstraction this provides is to separate the representation of data from the computation. Many current parallel programming models use a shared memory model to provide data abstraction but this doesn't scale well with large numbers of cores due to non-determinism and access latency. This paper proposes a simple programming model that allows scalable parallel programs to be expressed with distributed representations of data and it provides the programmer with the flexibility to employ shared or distributed styles of data-parallelism where applicable. It is capable of an efficient implementation, and with the provision of a small set of primitive capabilities in the hardware, it can be compiled to operate directly on the hardware, in the same way stack-based allocation operates for subroutines in sequential machines.