Microbenchmark-Driven Analytical Performance Modeling Across Modern GPU Architectures

TL;DR

This paper presents microbenchmark-based analytical performance models for modern GPUs, achieving high accuracy across different architectures and validating with real benchmarks, with open-source release planned.

Contribution

It introduces detailed performance models for NVIDIA Blackwell and AMD CDNA3 GPUs based on systematic microbenchmark characterization, improving accuracy over naive models.

Findings

Models achieve 1.31% and 0.09% MAE on Blackwell and MI300A.

Naive roofline baselines exceed 95% error on the same kernels.

Models are adaptable to other GPU architectures with minimal restructuring.

Abstract

Rapidly evolving GPU architectures featuring complex memory hierarchies, matrix units, and varied precision formats continue to widen the gap between theoretical peaks and achievable performance. We design and develop analytical performance models for NVIDIA Blackwell (B200) and AMD CDNA3 (MI300A) grounded in systematic microbenchmark characterization. For Blackwell, the model captures Tensor Memory (TMEM), asynchronous bulk copy (TMA), and 5th-generation tensor cores; for CDNA3, the model captures Infinity Cache hierarchy, VGPR constraints, and occupancy. Validation yields 1.31% MAE on B200 (21 kernels) and 0.09% on MI300A (27 kernels), while naive roofline baselines exceed 95% error on the same kernels. We further validate the models using Rodinia~3.1 and SPEChpc 2021 Tiny.The models are updated with HBM bandwidth, capacity, and cache parameters and applied to H200 (Hopper) and MI250X (CDNA2), indicating no major restructuring of the models are needed. All models and benchmarks will be released as open-source upon acceptance.