Accelerating Lattice QCD Simulations using GPUs

TL;DR

This paper explores GPU acceleration for lattice QCD simulations, specifically improving the DDalphaAMG solver, achieving a 3x speedup for large lattices, and discusses challenges and potential future improvements.

Contribution

It introduces a new memory access scheme and evaluates GPU acceleration for DDalphaAMG, demonstrating significant speedups for large-scale lattice QCD problems.

Findings

Achieved around 3x speedup for large lattices using GPUs.

Memory-bound implementation limits acceleration benefits.

Large-scale problems are necessary for GPU advantages over CPUs.

Abstract

Solving discretized versions of the Dirac equation represents a large share of execution time in lattice Quantum Chromodynamics (QCD) simulations. Many high-performance computing (HPC) clusters use graphics processing units (GPUs) to offer more computational resources. Our solver program, DDalphaAMG, previously was unable to fully take advantage of GPUs to accelerate its computations. Making use of GPUs for DDalphaAMG is an ongoing development, and we will present some current progress herein. Through a detailed description of our development, this thesis should offer valuable insights into using GPUs to accelerate a memory-bound CPU implementation. We developed a storage scheme for multiple tuples, which allows much more efficient memory access on GPUs, given that the element at the same index is read from multiple tuples simultaneously. Still, our implementation of a discrete Dirac operator is memory-bound, and we only achieved improvements for large linear systems on few nodes at the JUWELS cluster. These improvements do not currently overcome additional introduced overheads. However, the results for the application of the Wilson-Dirac operator show a speedup of around 3 for large lattices. If the additional overheads can be eliminated in the future, GPUs could reduce the DDalphaAMG execution time significantly for large lattices. We also found that a previous publication on the GPU acceleration of DDalphaAMG, underrepresented the achieved speedup, because small lattices were used. This further highlights that GPUs often require large-scale problems to solve in order to be faster than CPUs