Exploration of Unary Arithmetic-Based Matrix Multiply Units for Low Precision DL Accelerators

TL;DR

This paper evaluates unary GEMM designs for low precision deep learning inference, comparing them to binary GEMM, and explores their energy efficiency and optimal use cases in edge AI accelerators.

Contribution

It provides a comprehensive evaluation of three recent unary GEMM proposals, analyzing their tradeoffs and potential for energy-efficient low precision DL inference.

Findings

Unary GEMM designs show promising energy efficiency for low precision DL.

Optimal bit-widths and matrix sizes vary across designs, indicating specific use-case advantages.

Unary GEMM can be effectively integrated into future edge AI accelerators.

Abstract

General matrix multiplication (GEMM) is a fundamental operation in deep learning (DL). With DL moving increasingly toward low precision, recent works have proposed novel unary GEMM designs as an alternative to conventional binary GEMM hardware. A rigorous evaluation of recent unary and binary GEMM designs is needed to assess the potential of unary hardware for future DL compute. This paper focuses on unary GEMM designs for integer-based DL inference and performs a detailed evaluation of three latest unary design proposals, namely, uGEMM, tuGEMM and tubGEMM, by comparing them to a conventional binary GEMM. Rigorous post-synthesis evaluations beyond prior works are performed across varying bit-widths and matrix sizes to assess the designs' tradeoffs and determine optimal sweetspots. Further, we perform weight sparsity analysis across eight pretrained convolutional neural networks (CNNs) and the LLaMA2 large language model (LLM). In this work, we demonstrate how unary GEMM can be effectively used for energy-efficient compute in future edge AI accelerators.