Scaling of a Fast Fourier Transform and a Pseudo-spectral Fluid Solver up to 196608 cores

TL;DR

This paper evaluates the scalability of FFT and pseudospectral fluid solvers on large supercomputers, demonstrating near-linear scaling up to hundreds of thousands of cores and analyzing communication bottlenecks.

Contribution

It provides detailed scaling analysis of FFTK and Tarang on large-scale supercomputers, highlighting communication challenges and performance characteristics.

Findings

Communication dominates computation at large scales.

Computation scales as p^{-1}, communication as n^{-eta}.

Near-ideal weak and strong scaling up to 196608 cores.

Abstract

In this paper we present scaling results of a FFT library, FFTK, and a pseudospectral code, Tarang, on grid resolutions up to $8192^3$ grid using 65536 cores of Blue Gene/P and 196608 cores of Cray XC40 supercomputers. We observe that communication dominates computation, more so on the Cray XC40. The computation time scales as $T_\mathrm{comp} \sim p^{-1}$, and the communication time as $T_\mathrm{comm} \sim n^{-\gamma_2}$ with $\gamma_2$ ranging from 0.7 to 0.9 for Blue Gene/P, and from 0.43 to 0.73 for Cray XC40. FFTK, and the fluid and convection solvers of Tarang exhibit weak as well as strong scaling nearly up to 196608 cores of Cray XC40. We perform a comparative study of the performance on the Blue Gene/P and Cray XC40 clusters.