A mathematical model of asynchronous data flow in parallel computers

TL;DR

This paper develops a continuum mathematical model of data flow in parallel computers, using differential equations and fluid dynamics concepts, to analyze and predict the impact of processing capabilities on data transfer and computation progress.

Contribution

It introduces a novel continuum model derived from discrete data flow, with proven solution properties and validated through numerical experiments.

Findings

Qualitative agreement between discrete and continuum models

Processing capability variations significantly affect data flow progress

Mathematical framework facilitates analysis of asynchronous data transfer

Abstract

We present a simplified model of data flow on processors in a high performance computing framework involving computations necessitating inter-processor communications. From this ordinary differential model, we take its asymptotic limit, resulting in a model which treats the computer as a continuum of processors and data flow as an Eulerian fluid governed by a conservation law. We derive a Hamilton-Jacobi equation associated with this conservation law for which the existence and uniqueness of solutions can be proven. We then present the results of numerical experiments for both discrete and continuum models; these show a qualitative agreement between the two and the effect of variations in the computing environment's processing capabilities on the progress of the modeled computation.