Generalized Hyper-Systolic Algorithm

TL;DR

This paper generalizes the hyper-systolic algorithm for parallel computing, reducing communication complexity from linear to square root scale, and demonstrates its effectiveness with parallel matrix multiplication on a Cray-T3D.

Contribution

It introduces a generalized hyper-systolic algorithm that improves communication efficiency for massive parallel data processing.

Findings

Communication complexity is proportional to √n_p V

Implementation details are provided for the generalized algorithm

Parallel matrix multiplication benefits from the new algorithm

Abstract

We generalize the hyper-systolic algorithm proposed in [1] for abstract data structures on massive parallel computers with $n_p$ processors. For a problem of size $V$ the communication complexity of the hyper-systolic algorithm is proportional to $\sqrt{n_p}V$, to be compared with $n_pV$ for the systolic case. The implementation technique is explained in detail and the example of the parallel matrix-matrix multiplication is tested on the Cray-T3D.