Performance Engineering of the Kernel Polynomial Method on Large-Scale CPU-GPU Systems

TL;DR

This paper demonstrates how to optimize and implement the Kernel Polynomial Method efficiently on large-scale CPU-GPU systems, enabling high-performance spectral analysis in quantum physics and chemistry at petascale levels.

Contribution

It introduces a set of optimization techniques and a hybrid-parallel framework for scalable KPM implementations on heterogeneous CPU-GPU architectures.

Findings

Achieved decoupling of sparse matrix computations from memory bandwidth limitations.

Enabled pure data streaming through combined outer and inner iterations.

Successfully performed large-scale electronic structure calculations on petascale systems.

Abstract

The Kernel Polynomial Method (KPM) is a well-established scheme in quantum physics and quantum chemistry to determine the eigenvalue density and spectral properties of large sparse matrices. In this work we demonstrate the high optimization potential and feasibility of peta-scale heterogeneous CPU-GPU implementations of the KPM. At the node level we show that it is possible to decouple the sparse matrix problem posed by KPM from main memory bandwidth both on CPU and GPU. To alleviate the effects of scattered data access we combine loosely coupled outer iterations with tightly coupled block sparse matrix multiple vector operations, which enables pure data streaming. All optimizations are guided by a performance analysis and modelling process that indicates how the computational bottlenecks change with each optimization step. Finally we use the optimized node-level KPM with a hybrid-parallel framework to perform large scale heterogeneous electronic structure calculations for novel topological materials on a petascale-class Cray XC30 system.