Performance Optimization of SU3_Bench on Xeon and Programmable Integrated Unified Memory Architecture

TL;DR

This paper explores optimizing the SU3_Bench microbenchmark across Xeon and Intel PIUMA architectures, addressing performance challenges and demonstrating a twofold performance boost on Xeon, while analyzing the impact of architecture-specific factors.

Contribution

It identifies key challenges in achieving peak performance of SU3_Bench on Xeon and PIUMA, and proposes optimization strategies tailored to each architecture's characteristics.

Findings

Performance on Xeon improved by 2x with optimizations.

Performance on PIUMA influenced more by pipeline throughput than bandwidth.

Comparison shows Xeon has about ten times more flops-per-byte than PIUMA.

Abstract

SU3\_Bench is a microbenchmark developed to explore performance portability across multiple programming models/methodologies using a simple, but nontrivial, mathematical kernel. This kernel has been derived from the MILC lattice quantum chromodynamics (LQCD) code. SU3\_Bench is bandwidth bound and generates regular compute and data access patterns. Therefore, on most traditional CPU and GPU-based systems, its performance is mainly determined by the achievable memory bandwidth. Although SU3\_Bench is a simple kernel, experience says its subtleties require a certain amount of tweaking to achieve peak performance for a given programming model and hardware, making performance portability challenging. In this paper, we share some of the challenges in obtaining the peak performance for SU3\_Bench on a state-of-the-art Intel Xeon machine, due to the nuances of variable definition, the nature of compiler-provided default constructors, how memory is accessed at object creation time, and the NUMA effects on the machine. We discuss how to tackle those challenges to improve SU3\_Bench's performance by \(2\times\) compared to the original OpenMP implementation available at Github. This provides a valuable lesson for other similar kernels. Expanding on the performance portability aspects, we also show early results obtained porting SU3\_Bench to the new Intel Programmable Integrated Unified Memory Architecture (PIUMA), characterized by a more balanced flops-to-byte ratio. This paper shows that it is not the usual bandwidth or flops, rather the pipeline throughput, that determines SU3\_Bench's performance on PIUMA. Finally, we show how to improve performance on PIUMA and how that compares with the performance on Xeon, which has around one order of magnitude more flops-per-byte.