Bringing Auto-tuning to HIP: Analysis of Tuning Impact and Difficulty on AMD and Nvidia GPUs

TL;DR

This paper introduces an auto-tuner for AMD's HIP, analyzing its impact and difficulty, and compares performance improvements on AMD and Nvidia GPUs, highlighting the importance of GPU-specific tuning.

Contribution

It extends Kernel Tuner for AMD HIP, providing the first detailed analysis of auto-tuning impact and difficulty on AMD GPUs compared to Nvidia.

Findings

Auto-tuning yields 10x performance improvement on AMD GPUs.

Auto-tuning impact is higher on AMD than Nvidia.

Nvidia-tuned applications perform poorly on AMD GPUs without auto-tuning.

Abstract

Many studies have focused on developing and improving auto-tuning algorithms for Nvidia Graphics Processing Units (GPUs), but the effectiveness and efficiency of these approaches on AMD devices have hardly been studied. This paper aims to address this gap by introducing an auto-tuner for AMD's HIP. We do so by extending Kernel Tuner, an open-source Python library for auto-tuning GPU programs. We analyze the performance impact and tuning difficulty for four highly-tunable benchmark kernels on four different GPUs: two from Nvidia and two from AMD. Our results demonstrate that auto-tuning has a significantly higher impact on performance on AMD compared to Nvidia (10x vs 2x). Additionally, we show that applications tuned for Nvidia do not perform optimally on AMD, underscoring the importance of auto-tuning specifically for AMD to achieve high performance on these GPUs.