Optimising Performance Through Unbalanced Decompositions

TL;DR

This paper improves the efficiency of gyrokinetic simulations by optimizing data decompositions, using unbalanced layouts for non-linear calculations, leading to up to 15% performance gains.

Contribution

It introduces an optimized, unbalanced data decomposition method for non-linear parts of gyrokinetic simulations, enhancing performance without disrupting linear calculation efficiency.

Findings

Eliminated communications in non-linear calculations.

Achieved up to 15% performance improvement.

Validated the approach on representative simulations.

Abstract

GS2 is an initial value gyrokinetic simulation code developed to study low-frequency turbulence in magnetized plasma. It is parallelised using MPI with the simulation domain decomposed across tasks. The optimal domain decomposition is non-trivial, and complicated by the different requirements of the linear and non-linear parts of the calculations. GS2 users currently choose a data layout, and are guided towards processor count that are efficient for linear calculations. These choices can, however, lead to data decompositions that are relatively inefficient for the non-linear calculations. We have analysed the performance impact of the data decompositions on the non-linear calculation and associated communications. This has helped us to optimise the decomposition algorithm by using unbalanced data layouts for the non-linear calculations whilst maintaining the existing decompositions for the linear calculations, which has completely eliminated communications for parts of the non-linear simulation and improved performance by up to 15% for a representative simulation.