Exploring metrics for analyzing dynamic behavior in MPI programs via a coupled-oscillator model

TL;DR

This paper introduces a physics-inspired coupled-oscillator model to analyze MPI program dynamics, capturing synchronization, delays, and bottlenecks, and providing new metrics for performance evaluation.

Contribution

It presents a novel coupled-oscillator framework based on the Kuramoto model for modeling MPI program behavior, incorporating topology-aware interactions and stochastic noise.

Findings

Simulations qualitatively match MPI trace data

Moderate noise accelerates resynchronization

Metrics effectively evaluate phase coherence and disruption

Abstract

We propose a novel, lightweight, and physically inspired approach to modeling the dynamics of parallel distributed-memory programs. Inspired by the Kuramoto model, we represent MPI processes as coupled oscillators with topology-aware interactions, custom coupling potentials, and stochastic noise. The resulting system of nonlinear ordinary differential equations opens a path to modeling key performance phenomena of parallel programs, including synchronization, delay propagation and decay, bottlenecks, and self-desynchronization. This paper introduces interaction potentials to describe memory- and compute-bound workloads and employs multiple quantitative metrics -- such as an order parameter, synchronization entropy, phase gradients, and phase differences -- to evaluate phase coherence and disruption. We also investigate the role of local noise and show that moderate noise can accelerate resynchronization in scalable applications. Our simulations align qualitatively with MPI trace data, showing the potential of physics-informed abstractions to predict performance patterns, which offers a new perspective for performance modeling and software-hardware co-design in parallel computing.