Parallelization of Network Dynamics Computations in Heterogeneous Distributed Environment

TL;DR

This paper introduces a transparent parallelization method for large network dynamics simulations on heterogeneous systems, enabling near-linear speed-up without requiring end-user parallel programming.

Contribution

It presents a novel, user-friendly parallelization approach that efficiently utilizes heterogeneous computing resources for simulating large non-linear dynamical networks.

Findings

Achieved near-linear speed-up for networks with up to 10^8 elements.

Demonstrated effectiveness on Hodgkin-Huxley, FitzHugh-Nagumo, and Kuramoto models.

Applicable to various hardware including GPUs and multiprocessors.

Abstract

This paper addresses the problem of parallelizing computations to study non-linear dynamics in large networks of non-locally coupled oscillators using heterogeneous computing resources. The proposed approach can be applied to a variety of non-linear dynamics models with runtime specification of parameters and network topologies. Parallelizing the solution of equations for different network elements is performed transparently and, in contrast to available tools, does not require parallel programming from end-users. The runtime scheduler takes into account the performance of computing and communication resources to reduce downtime and to achieve a quasi-optimal parallelizing speed-up. The proposed approach was implemented, and its efficiency is proven by numerous applications for simulating large dynamical networks with 10^3-10^8 elements described by Hodgkin-Huxley, FitzHugh-Nagumo, and Kuramoto models, for investigating pathological synchronization during Parkinson's disease, analyzing multi-stability, for studying chimera and solitary states in 3D networks, etc. All the above computations may be performed using symmetrical multiprocessors, graphic processing units, and a network of workstations within the same run and it was demonstrated that near-linear speed-up can be achieved for large networks. The proposed approach is promising for extension to new hardware like edge-computing devices.