Distributed N-body Simulation on the Grid Using Dedicated Hardware

TL;DR

This paper evaluates the performance of large-scale gravitational N-body simulations on a global grid with specialized hardware, developing a model to predict performance and demonstrating feasibility for future large simulations.

Contribution

It introduces a performance model for N-body simulations on distributed grids with specialized hardware, enabling prediction of execution times across different infrastructures.

Findings

High network bandwidth dominates simulation performance over latency.

Adding GRAPE hardware significantly reduces computation time.

Predicted simulation of a few million particles can be completed in about ten hours.

Abstract

We present performance measurements of direct gravitational N -body simulation on the grid, with and without specialized (GRAPE-6) hardware. Our inter-continental virtual organization consists of three sites, one in Tokyo, one in Philadelphia and one in Amsterdam. We run simulations with up to 196608 particles for a variety of topologies. In many cases, high performance simulations over the entire planet are dominated by network bandwidth rather than latency. With this global grid of GRAPEs our calculation time remains dominated by communication over the entire range of N, which was limited due to the use of three sites. Increasing the number of particles will result in a more efficient execution. Based on these timings we construct and calibrate a model to predict the performance of our simulation on any grid infrastructure with or without GRAPE. We apply this model to predict the simulation performance on the Netherlands DAS-3 wide area computer. Equipping the DAS-3 with GRAPE-6Af hardware would achieve break-even between calculation and communication at a few million particles, resulting in a compute time of just over ten hours for 1 N -body time unit. Key words: high-performance computing, grid, N-body simulation, performance modelling