Systolic Array Data Flows for Efficient Matrix Multiplication in Deep Neural Networks

TL;DR

This paper analyzes different systolic array data flows for matrix multiplication in deep neural networks, demonstrating how choosing the optimal data flow can significantly reduce energy consumption in AI hardware accelerators.

Contribution

It provides a comparative analysis of three main systolic array data flows and their energy efficiency across various matrix sizes using simulation.

Findings

Weight Stationary often reduces energy for large matrices

Input Stationary is more efficient for smaller matrices

Selecting the right data flow optimizes energy use in DNN accelerators

Abstract

The paper discusses how Systolic Arrays can improve matrix multiplication for deep neural networks (DNNs). With AI models like OpenAI's GPT now containing trillions of parameters, the need for efficient matrix multiplication is more critical than ever. In this paper, the three main systolic array data flows: Weight Stationary (WS), Input Stationary (IS), and Output Stationary (OS) are discussed. Each data flow's energy consumption and efficiency across various matrix sizes are calculated using the SCALE-Sim simulator. The results show that selecting the right data flow for specific matrix configurations can drastically reduce energy consumption. The conclusions provide helpful insights into optimizing hardware for AI and machine learning applications, offering potential improvements in designing energy-efficient DNN accelerators.