DBCSR: A Library for Dense Matrix Multiplications on Distributed GPU-Accelerated Systems

TL;DR

The paper introduces DBCSR, a library optimized for dense matrix multiplication on distributed GPU systems, outperforming existing vendor libraries by up to 2.5 times in speed.

Contribution

It presents a novel library, DBCSR, optimized for dense matrix multiplication on distributed GPU-accelerated systems, filling a gap in existing high-performance linear algebra libraries.

Findings

DBCSR outperforms vendor-optimized GPU ScaLAPACK by up to 2.5x.

The library is optimized for dense matrix multiplication on distributed GPU systems.

Average performance improvement of 1.4x over existing solutions.

Abstract

Most, if not all the modern scientific simulation packages utilize matrix algebra operations. Among the operation of the linear algebra, one of the most important kernels is the multiplication of matrices, dense and sparse. Examples of application of such a kernel are in electronic structure calculations, machine learning, data mining, graph processing, and digital signal processing. Several optimized libraries exist that can achieve high-performance on distributed systems. Only a few of them target distributed GPU-accelerated systems. In most of the cases, these libraries are provided and optimized by system vendors for their specific computer systems. In this paper, we present the DBCSR library (Distributed Block Compressed Sparse Row) for the distributed dense matrix-matrix multiplications. Although the library is specifically designed for block-sparse matrix-matrix multiplications, we optimized it for the dense case on GPU-accelerated systems. We show that the DBCSR outperforms the multiplication of matrices of different sizes and shapes provided by a vendor optimized GPU version of the ScaLAPACK library up to 2.5x (1.4x on average).