Cascaded Prediction and Asynchronous Execution of Iterative Algorithms on Heterogeneous Platforms

TL;DR

This paper presents a machine learning-based cascaded prediction method combined with asynchronous execution to optimize sparse matrix-vector multiplication and iterative algorithms on heterogeneous platforms, significantly improving performance.

Contribution

It introduces a novel cascaded prediction approach and an asynchronous execution model that together reduce overheads and accelerate computations in sparse matrix operations.

Findings

SpMV acceleration by 1.33x on average

Iterative algorithm optimization by 2.55x on average

Effective mitigation of preprocessing overheads

Abstract

Owing to the diverse scales and varying distributions of sparse matrices arising from practical problems, a multitude of choices are present in the design and implementation of sparse matrix-vector multiplication (SpMV). Researchers have proposed many machine learning-based optimization methods for SpMV. However, these efforts only support one area of sparse matrix format selection, SpMV algorithm selection, or parameter configuration, and rarely consider a large amount of time overhead associated with feature extraction, model inference, and compression format conversion. This paper introduces a machine learning-based cascaded prediction method for SpMV computations that spans various computing stages and hierarchies. Besides, an asynchronous and concurrent computing model has been designed and implemented for runtime model prediction and iterative algorithm solving on heterogeneous computing platforms. It not only offers comprehensive support for the iterative algorithm-solving process leveraging machine learning technology, but also effectively mitigates the preprocessing overheads. Experimental results demonstrate that the cascaded prediction introduced in this paper accelerates SpMV by 1.33x on average, and the iterative algorithm, enhanced by cascaded prediction and asynchronous execution, optimizes by 2.55x on average.