Loading Large Sparse Matrices Stored in Files in the Adaptive-Blocking Hierarchical Storage Format

TL;DR

This paper introduces a parallel algorithm for efficiently loading large sparse matrices stored in the adaptive-blocking hierarchical storage format into distributed HPC memory, accommodating various process mappings and storage formats.

Contribution

The paper presents a novel parallel loading algorithm specifically designed for matrices in ABHSF, compatible with different process configurations and storage formats.

Findings

Algorithm performs efficiently in HPC environments

Compatible with various process mappings and storage formats

Empirical evaluation demonstrates effectiveness

Abstract

The parallel algorithm for loading large sparse matrices from files into distributed memories of high performance computing (HPC) systems is presented. This algorithm was designed specially for matrices stored in files in the space-effcient adaptive-blocking hierarchical storage format (ABHSF). The algorithm can be used even if matrix storing and loading procedures use a different number of processes, different matrix-processes mapping, or different in-memory storage format. The file format based on the utilization of the HDF5 library is described as well. Finally, the presented experimental study evaluates the proposed algorithm empirically.