Double-Precision Matrix Multiplication Emulation via Ozaki-II Scheme with FP8 Quantization

TL;DR

This paper introduces a novel FP8-based Ozaki-II scheme for emulating double-precision matrix multiplication, improving efficiency and performance on modern architectures with reduced INT8 performance.

Contribution

It extends the Ozaki-II scheme to utilize FP8 MMA units, enabling efficient DGEMM emulation on architectures with limited INT8 performance.

Findings

FP8-based Ozaki-II scheme reduces computational cost.

Enables efficient FP64 emulation on modern architectures.

Addresses performance issues in recent GPU architectures.

Abstract

In this paper, we propose a method for emulating double-precision general matrix--matrix multiplication (DGEMM), a fundamental and performance-critical kernel in many high-performance computing applications. Ozaki-I and Ozaki-II are established DGEMM emulation schemes via low-precision matrix multiply-accumulate (MMA) units. For the Ozaki-I scheme, INT8-, FP8-, and FP16-based implementations have been proposed, all of which can be realized based on the same underlying algorithmic structure. In contrast, although INT8-based implementations of the Ozaki-II scheme have been reported, the original algorithm cannot be directly adapted to exploit FP8 MMA units. In several recent architectures, such as NVIDIA Blackwell Ultra and NVIDIA Rubin, INT8 performance has been reduced, making reliance on INT8 alone insufficient. Therefore, we introduce a novel technique to demonstrate DGEMM emulation based on the Ozaki-II scheme that operates on FP8 MMA units. Compared to the FP8-based Ozaki-I scheme, our method significantly reduces the computational cost and enables efficient FP64 emulation.