From Merging Frameworks to Merging Stars: Experiences using HPX, Kokkos and SIMD Types

TL;DR

This paper discusses integrating SIMD types into Octo-Tiger, a star merger simulation code, to improve performance portability across diverse hardware, and evaluates the resulting speedups and scaling issues.

Contribution

It introduces the use of std::experimental::simd and a new SVE backend in Kokkos for Octo-Tiger, enhancing SIMD support on A64FX CPUs and improving performance.

Findings

Good SIMD speedup on A64FX CPU

Noticeable speedups on Intel Icelake and AMD EPYC CPUs

Scaling issues observed on AMD EPYC CPU

Abstract

Octo-Tiger, a large-scale 3D AMR code for the merger of stars, uses a combination of HPX, Kokkos and explicit SIMD types, aiming to achieve performance-portability for a broad range of heterogeneous hardware. However, on A64FX CPUs, we encountered several missing pieces, hindering performance by causing problems with the SIMD vectorization. Therefore, we add std::experimental::simd as an option to use in Octo-Tiger's Kokkos kernels alongside Kokkos SIMD, and further add a new SVE (Scalable Vector Extensions) SIMD backend. Additionally, we amend missing SIMD implementations in the Kokkos kernels within Octo-Tiger's hydro solver. We test our changes by running Octo-Tiger on three different CPUs: An A64FX, an Intel Icelake and an AMD EPYC CPU, evaluating SIMD speedup and node-level performance. We get a good SIMD speedup on the A64FX CPU, as well as noticeable speedups on the other two CPU platforms. However, we also experience a scaling issue on the EPYC CPU.