OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling

TL;DR

OpenGeMM is an open-source GeMM accelerator platform that combines high efficiency, high utilization, and programmability through a lightweight RISC-V control and tight memory coupling, optimized for neural network workloads.

Contribution

It introduces a configurable GeMM accelerator with RISC-V control and novel memory access strategies, achieving high utilization and efficiency for neural network acceleration.

Findings

Achieves 81.89% to 99.34% hardware utilization across CNN and Transformer workloads.

Demonstrates 3.58x to 16.40x speedup over state-of-the-art open-source accelerators.

Attains 4.68 TOPS/W system efficiency.

Abstract

Deep neural networks (DNNs) face significant challenges when deployed on resource-constrained extreme edge devices due to their computational and data-intensive nature. While standalone accelerators tailored for specific application scenarios suffer from inflexible control and limited programmability, generic hardware acceleration platforms coupled with RISC-V CPUs can enable high reusability and flexibility, yet typically at the expense of system level efficiency and low utilization. To fill this gap, we propose OpenGeMM, an open-source acceleration platform, jointly demonstrating high efficiency and utilization, as well as ease of configurability and programmability. OpenGeMM encompasses a parameterized Chisel-coded GeMM accelerator, a lightweight RISC-V processor, and a tightly coupled multi-banked scratchpad memory. The GeMM core utilization and system efficiency are boosted through three mechanisms: configuration pre-loading, input pre-fetching with output buffering, and programmable strided memory access. Experimental results show that OpenGeMM can consistently achieve hardware utilization ranging from 81.89% to 99.34% across diverse CNN and Transformer workloads. Compared to the SotA open-source Gemmini accelerator, OpenGeMM demonstrates a 3.58x to 16.40x speedup on normalized throughput across a wide variety ofGeMM workloads, while achieving 4.68 TOPS/W system efficiency.