Process-Oriented Parallel Programming with an Application to Data-Intensive Computing

TL;DR

This paper proposes process-oriented programming as a natural, object-based approach to parallel computing, enabling high-level, minimal-syntax parallelism in languages like C++ and Python, demonstrated through a large-scale Fourier transform application.

Contribution

It introduces process-oriented programming as a simple, object-based extension for parallel programming, bridging the gap between object-oriented design and high-performance parallel computation.

Findings

Achieved high data throughput in a 3D Fourier transform on commodity hardware.

Process-oriented code is concise, only a few hundred lines.

Demonstrated natural object-based parallelism with minimal syntax extension.

Abstract

We introduce process-oriented programming as a natural extension of object-oriented programming for parallel computing. It is based on the observation that every class of an object-oriented language can be instantiated as a process, accessible via a remote pointer. The introduction of process pointers requires no syntax extension, identifies processes with programming objects, and enables processes to exchange information simply by executing remote methods. Process-oriented programming is a high-level language alternative to multithreading, MPI and many other languages, environments and tools currently used for parallel computations. It implements natural object-based parallelism using only minimal syntax extension of existing languages, such as C++ and Python, and has therefore the potential to lead to widespread adoption of parallel programming. We implemented a prototype system for running processes using C++ with MPI and used it to compute a large three-dimensional Fourier transform on a computer cluster built of commodity hardware components. Three-dimensional Fourier transform is a prototype of a data-intensive application with a complex data-access pattern. The process-oriented code is only a few hundred lines long, and attains very high data throughput by achieving massive parallelism and maximizing hardware utilization.