Annotation-guided AoS-to-SoA conversions and GPU offloading with data views in C++

TL;DR

This paper introduces a C++ language extension with annotations that enable automatic selection and conversion between array-of-structs and structure-of-arrays data layouts, optimizing performance across CPU and GPU architectures.

Contribution

It proposes a novel C++ extension allowing developers to guide memory layout choices, with automatic data conversions handled by the compiler for improved performance.

Findings

Effective data layout optimization demonstrated on SPH code

Performance improvements observed across CPU and GPU

Annotations challenge traditional performance engineering approaches

Abstract

The C++ programming language provides classes and structs as fundamental modeling entities. Consequently, C++ code tends to favour array-of-structs (AoS) for encoding data sequences, even though structure-of-arrays (SoA) yields better performance for some calculations. We propose a C++ language extension based on attributes that allows developers to guide the compiler in selecting memory arrangements, i.e.~to select the optimal choice between AoS and SoA dynamically depending on both the execution context and algorithm step. The compiler can then automatically convert data into the preferred format prior to the calculations and convert results back afterward. The compiler handles all the complexity of determining which data to convert and how to manage data transformations. Our implementation realises the compiler-extension for the new annotations in Clang and demonstrates their effectiveness through a smoothed particle hydrodynamics (SPH) code, which we evaluate on an Intel CPU, an ARM CPU, and a Grace-Hopper GPU. While the separation of concerns between data structure and operators is elegant and provides performance improvements, the new annotations do not eliminate the need for performance engineering. Instead, they challenge conventional performance wisdom and necessitate rethinking approaches how to write efficient implementations.