Multi GPU Performance of Conjugate Gradient Solver with Staggered Fermions in Mixed Precision

TL;DR

This paper demonstrates a multi-GPU conjugate gradient solver optimized with mixed precision to leverage GPU speed advantages, achieving significant performance improvements in scientific computations.

Contribution

The paper introduces a mixed precision algorithm for multi-GPU CG solvers that enhances performance by reducing memory usage and increasing computation speed.

Findings

Single precision is 8 times faster than double precision.

Performance reaches 145 GFLOPS per GPU without communication overhead.

Including network communication, performance is 36 GFLOPS per GPU.

Abstract

GPU has a significantly higher performance in single-precision computing than that of double precision. Hence, it is important to take a maximal advantage of the single precision in the CG inverter, using the mixed precision method. We have implemented mixed precision algorithm to our multi GPU conjugate gradient solver. The single precision calculation use half of the memory that is used by the double precision calculation, which allows twice faster data transfer in memory I/O. In addition, the speed of floating point calculations is 8 times faster in single precision than in double precision. The overall performance of our CUDA code for CG is 145 giga flops per GPU (GTX480), which does not include the infiniband network communication. If we include the infiniband communication, the overall performance is 36 giga flops per GPU (GTX480).