High Performance Gravitational N-body Simulations on a Planet-wide Distributed Supercomputer

TL;DR

This paper demonstrates the feasibility of running large-scale gravitational N-body simulations across multiple globally distributed supercomputers, achieving high performance despite network latency.

Contribution

It presents the first performance analysis of cosmological N-body simulations on a planet-wide distributed supercomputer network.

Findings

Achieved 87% performance compared to single supercomputer setups.

Successfully ran simulations across four continents with high efficiency.

Identified local scheduling policies as main obstacle to scalability.

Abstract

We report on the performance of our cold-dark matter cosmological N-body simulation which was carried out concurrently using supercomputers across the globe. We ran simulations on 60 to 750 cores distributed over a variety of supercomputers in Amsterdam (the Netherlands, Europe), in Tokyo (Japan, Asia), Edinburgh (UK, Europe) and Espoo (Finland, Europe). Regardless the network latency of 0.32 seconds and the communication over 30.000 km of optical network cable we are able to achieve about 87% of the performance compared to an equal number of cores on a single supercomputer. We argue that using widely distributed supercomputers in order to acquire more compute power is technically feasible, and that the largest obstacle is introduced by local scheduling and reservation policies.