Efficient executions of Pipelined Conjugate Gradient Method on Heterogeneous Architectures

TL;DR

This paper presents three novel methods for efficiently executing the Pipelined Conjugate Gradient algorithm on heterogeneous CPU-GPU architectures, achieving significant speedups over existing CPU and GPU implementations.

Contribution

It introduces task-parallelism and data parallelism strategies, including a performance model-based workload decomposition, for the Pipelined PCG method on heterogeneous systems.

Findings

Up to 8x speedup over CPU implementations

Up to 5x speedup over GPU implementations

Effective handling of large problems exceeding GPU memory

Abstract

The Preconditioned Conjugate Gradient (PCG) method is widely used for solving linear systems of equations with sparse matrices. A recent version of PCG, Pipelined PCG, eliminates the dependencies in the computations of the PCG algorithm so that the non-dependent computations can be overlapped with communication. In this paper, we propose three methods for efficient execution of the Pipelined PCG algorithm on GPU accelerated heterogeneous architectures. The first two methods achieve task-parallelism using asynchronous executions of different tasks on CPU cores and GPU. The third method achieves data parallelism by decomposing the workload between CPU and GPU based on a performance model. The performance model takes into account the relative performance of CPU cores and GPU using some initial executions and performs 2D data decomposition. We also implement optimization strategies like kernel fusion for GPU and merging vector operations for CPU. Our methods give up to 8x speedup and on average 3x speedup over PCG CPU implementation of Paralution and PETSc libraries. They also give up to 5x speedup and on average 1.45x speedup over PCG GPU implementation of Paralution and PETSc libraries. The third method also provides an efficient solution for solving problems that cannot be fit into the GPU memory and gives up to 2.5x speedup for such problems.