ML-Based Optimum Number of CUDA Streams for the GPU Implementation of the Tridiagonal Partition Method

TL;DR

This paper develops a heuristic and models to determine the optimal number of CUDA streams for GPU-based tridiagonal partition algorithms, improving performance prediction and optimization.

Contribution

It introduces a new heuristic, time complexity models, and an algorithm for predicting the optimal CUDA streams for GPU implementations of the partition method.

Findings

Empirical data validates the models and heuristic.

The algorithm predicts optimal CUDA streams with acceptable accuracy.

Regression models effectively estimate overhead and non-dominant operation times.

Abstract

This paper presents a heuristic for finding the optimum number of CUDA streams by using tools common to the modern AI-oriented approaches and applied to the parallel partition algorithm. A time complexity model for the GPU realization of the partition method is built. Further, a refined time complexity model for the partition algorithm being executed on multiple CUDA streams is formulated. Computational experiments for different SLAE sizes are conducted, and the optimum number of CUDA streams for each of them is found empirically. Based on the collected data a model for the sum of the times for the non-dominant GPU operations (that take part in the stream overlap) is formulated using regression analysis. A fitting non-linear model for the overhead time connected with the creation of CUDA streams is created. Statistical analysis is done for all the built models. An algorithm for finding the optimum number of CUDA streams is formulated. Using this algorithm, together with the two models mentioned above, predictions for the optimum number of CUDA streams are made. Comparing the predicted values with the actual data, the algorithm is deemed to be acceptably good.