FTTN: Feature-Targeted Testing for Numerical Properties of NVIDIA & AMD Matrix Accelerators

TL;DR

This paper introduces Feature Targeted Tests for Numerical Properties (FTTN) to identify numerical behaviors of NVIDIA and AMD matrix accelerators, aiding reliable code porting across diverse hardware.

Contribution

The paper develops a comprehensive suite of tests to reveal numerical behaviors of matrix accelerators, addressing a gap in publicly available documentation.

Findings

Different hardware produces varying matrix multiplication results.

FTTN tests reveal differences in rounding modes and precision handling.

Insights help improve code portability and reliability across platforms.

Abstract

NVIDIA Tensor Cores and AMD Matrix Cores (together called Matrix Accelerators) are of growing interest in high-performance computing and machine learning owing to their high performance. Unfortunately, their numerical behaviors are not publicly documented, including the number of extra precision bits maintained, the accumulation order of addition, and predictable subnormal number handling during computations. This makes it impossible to reliably port codes across these differing accelerators. This paper contributes a collection of {\em Feature Targeted Tests for Numerical Properties} that that help determine these features across five floating-point formats, four rounding modes and additional that highlight the rounding behaviors and preservation of extra precision bits. To show the practical relevance of FTTN, we design a simple matrix-multiplication test designed with insights gathered from our feature-tests. We executed this very simple test on five platforms, producing different answers: V100, A100, and MI250X produced 0, MI100 produced 255.875, and Hopper H100 produced 191.875. Our matrix multiplication tests employ patterns found in iterative refinement-based algorithms, highlighting the need to check for significant result variability when porting code across GPUs.