Evaluating the Performance Impact of Multiple Streams on the MIC-based Heterogeneous Platform

TL;DR

This paper systematically evaluates how multiple streams affect performance on Intel Xeon Phi, revealing overlaps in data transfer and kernel execution, and providing heuristics for optimizing task and resource granularity.

Contribution

It is the first comprehensive study on multiple streams performance impact on Xeon Phi, offering insights, experimental results, and heuristics for optimization.

Findings

Data transfers and kernel execution can be overlapped on Phi.

Performance improvements up to 24% with multiple streams.

Task and resource granularity significantly affect performance.

Abstract

Using \textit{multiple streams} can improve the overall system performance by mitigating the data transfer overhead on heterogeneous systems. Prior work focuses a lot on GPUs but little is known about the performance impact on (Intel Xeon) Phi. In this work, we apply multiple streams into six real-world applications on Phi. We then systematically evaluate the performance benefits of using multiple streams. The evaluation work is performed at two levels: the microbenchmarking level and the real-world application level. Our experimental results at the microbenchmark level show that data transfers and kernel execution can be overlapped on Phi, while data transfers in both directions are performed in a serial manner. At the real-world application level, we show that both overlappable and non-overlappable applications can benefit from using multiple streams (with an performance improvement of up to 24\%). We also quantify how task granularity and resource granularity impact the overall performance. Finally, we present a set of heuristics to reduce the search space when determining a proper task granularity and resource granularity. To conclude, our evaluation work provides lots of insights for runtime and architecture designers when using multiple streams on Phi.