# Low Overhead Instruction Latency Characterization for NVIDIA GPGPUs

## TL;DR

This paper presents a low-overhead, portable method for measuring instruction and memory latency in NVIDIA GPUs across multiple architectures, aiding hardware modeling and software optimization.

## Contribution

It introduces a novel, low-overhead analysis technique for exposing GPU instruction and memory latencies at the micro-architecture level.

## Key findings

- Characterized latencies across seven NVIDIA GPU architectures.
- Showed impact of CUDA compiler optimizations on latencies.
- Provided data to improve hardware models and software performance.

## Abstract

The last decade has seen a shift in the computer systems industry where heterogeneous computing has become prevalent. Graphics Processing Units (GPUs) are now present in supercomputers to mobile phones and tablets. GPUs are used for graphics operations as well as general-purpose computing (GPGPUs) to boost the performance of compute-intensive applications. However, the percentage of undisclosed characteristics beyond what vendors provide is not small. In this paper, we introduce a very low overhead and portable analysis for exposing the latency of each instruction executing in the GPU pipeline(s) and the access overhead of the various memory hierarchies found in GPUs at the micro-architecture level. Furthermore, we show the impact of the various optimizations the CUDA compiler can perform over the various latencies. We perform our evaluation on seven different high-end NVIDIA GPUs from five different generations/architectures: Kepler, Maxwell, Pascal, Volta, and Turing. The results in this paper can help architects to have an accurate characterization of the latencies of these GPUs, which will help in modeling the hardware accurately. Also, software developers can perform informed optimizations to their applications.