Physical Oscillator Model for Supercomputing

TL;DR

This paper introduces a physical oscillator model inspired by the Kuramoto Model to simulate and analyze the complex dynamical behaviors of parallel programs and hardware interactions in supercomputing environments.

Contribution

It extends the Kuramoto Model with new interaction potentials and sparse topologies to better represent resource-scalable and resource-bottlenecked parallel computing scenarios.

Findings

Model can mimic delay propagation in parallel processes.

Captures synchronization and desynchronization behaviors.

Differentiates between scalable and bottlenecked applications.

Abstract

A parallel program together with the parallel hardware it is running on is not only a vehicle to solve numerical problems, it is also a complex system with interesting dynamical behavior: resynchronization and desynchronization of parallel processes, propagating phases of idleness, and the peculiar effects of noise and system topology are just a few examples. We propose a physical oscillator model (POM) to describe aspects of the dynamics of interacting parallel processes. Motivated by the well-known Kuramoto Model, a process with its regular compute-communicate cycles is modeled as an oscillator which is coupled to other oscillators (processes) via an interaction potential. Instead of a simple all-to-all connectivity, we employ a sparse topology matrix mapping the communication structure and thus the inter-process dependencies of the program onto the oscillator model and propose two interaction potentials that are suitable for different scenarios in parallel computing: resource-scalable and resource-bottlenecked applications. The former are not limited by a resource bottleneck such as memory bandwidth or network contention, while the latter are. Unlike the original Kuramoto model, which has a periodic sinusoidal potential that is attractive for small angles, our characteristic potentials are always attractive for large angles and only differ in the short-distance behavior. We show that the model with appropriate potentials can mimic the propagation of delays and the synchronizing and desynchronizing behavior of scalable and bottlenecked parallel programs, respectively.