An ultra-low-power CGRA for accelerating Transformers at the edge

TL;DR

This paper presents an ultra-low-power CGRA architecture optimized for accelerating transformer models on edge devices by focusing on energy efficiency, data reuse, and minimal communication overhead.

Contribution

It introduces a novel CGRA design with dedicated processing and memory blocks, optimized for transformer GEMM operations in resource-constrained environments.

Findings

Achieves significant power reduction compared to traditional accelerators.

Demonstrates efficient GEMM computation suitable for edge transformer deployment.

Reduces memory bandwidth and latency through specialized interconnects.

Abstract

Transformers have revolutionized deep learning with applications in natural language processing, computer vision, and beyond. However, their computational demands make it challenging to deploy them on low-power edge devices. This paper introduces an ultra-low-power, Coarse-Grained Reconfigurable Array (CGRA) architecture specifically designed to accelerate General Matrix Multiplication (GEMM) operations in transformer models tailored for the energy and resource constraints of edge applications. The proposed architecture integrates a 4 x 4 array of Processing Elements (PEs) for efficient parallel computation and dedicated 4 x 2 Memory Operation Blocks (MOBs) for optimized LOAD/STORE operations, reducing memory bandwidth demands and enhancing data reuse. A switchless mesh torus interconnect network further minimizes power and latency by enabling direct communication between PEs and MOBs, eliminating the need for centralized switching. Through its heterogeneous array design and efficient dataflow, this CGRA architecture addresses the unique computational needs of transformers, offering a scalable pathway to deploy sophisticated machine learning models on edge devices.