Toward matrix multiplication for deep learning inference on the Xilinx Versal

TL;DR

This paper demonstrates that principles of general matrix multiplication (GEMM) can be effectively adapted to deep learning inference on Xilinx Versal's AI Engine, achieving near-peak performance with 16-bit integers.

Contribution

The paper shows how GEMM principles can be applied to AI Engine tiles in Xilinx Versal, achieving high-performance deep learning inference.

Findings

Achieves 86.7% of theoretical peak performance on AIE tiles.

Validates GEMM principles for deep learning on exotic accelerators.

Prototype implementation demonstrates practical viability.

Abstract

The remarkable positive impact of Deep Neural Networks on many Artificial Intelligence (AI) tasks has led to the development of various high performance algorithms as well as specialized processors and accelerators. In this paper we address this scenario by demonstrating that the principles underlying the modern realization of the general matrix multiplication (GEMM) in conventional processor architectures, are also valid to achieve high performance for the type of operations that arise in deep learning (DL) on an exotic accelerator such as the AI Engine (AIE) tile embedded in Xilinx Versal platforms. In particular, our experimental results with a prototype implementation of the GEMM kernel, on a Xilinx Versal VCK190, delivers performance close to 86.7% of the theoretical peak that can be expected on an AIE tile, for 16-bit integer operands.