Benchmarking the Nvidia GPU Lineage: From Early K80 to Modern A100 with Asynchronous Memory Transfers

TL;DR

This paper benchmarks Nvidia's A100 GPU against previous generations, revealing performance gains are less than expected on certain benchmarks, but can be improved with new asynchronous data-movement features.

Contribution

It provides a comprehensive empirical comparison of Nvidia GPU generations and demonstrates how to leverage new data-movement features to enhance performance.

Findings

A100 shows smaller performance gains than previous generations on some benchmarks.

Asynchronous data-movement features can mitigate performance issues.

Guidelines are provided for effectively using new data-movement features.

Abstract

For many, Graphics Processing Units (GPUs) provides a source of reliable computing power. Recently, Nvidia introduced its 9th generation HPC-grade GPUs, the Ampere 100, claiming significant performance improvements over previous generations, particularly for AI-workloads, as well as introducing new architectural features such as asynchronous data movement. But how well does the A100 perform on non-AI benchmarks, and can we expect the A100 to deliver the application improvements we have grown used to with previous GPU generations? In this paper, we benchmark the A100 GPU and compare it to four previous generations of GPUs, with particular focus on empirically quantifying our derived performance expectations, and -- should those expectations be undelivered -- investigate whether the introduced data-movement features can offset any eventual loss in performance? We find that the A100 delivers less performance increase than previous generations for the well-known Rodinia benchmark suite; we show that some of these performance anomalies can be remedied through clever use of the new data-movement features, which we microbenchmark and demonstrate where (and more importantly, how) they should be used.