Scalable and RISC-V Programmable Near-Memory Computing Architectures for Edge Nodes

TL;DR

This paper introduces scalable, RISC-V compatible near-memory computing architectures tailored for edge nodes, significantly enhancing energy efficiency and performance for AI/ML workloads in resource-constrained environments.

Contribution

It proposes a novel, flexible, and low-effort NMC architecture with two variants, NM-Caesar and NM-Carus, optimized for edge computing and integrated with RISC-V systems.

Findings

Up to 28x faster execution time compared to RISC-V CPU.

Up to 36x higher energy efficiency at system level.

NM-Carus achieves 306.7 GOPS/W in 8-bit matrix multiplications.

Abstract

The widespread adoption of data-centric algorithms, particularly Artificial Intelligence (AI) and Machine Learning (ML), has exposed the limitations of centralized processing infrastructures, driving a shift towards edge computing. This necessitates stringent constraints on energy efficiency, which traditional von Neumann architectures struggle to meet. The Compute-In-Memory (CIM) paradigm has emerged as a superior candidate due to its efficient exploitation of available memory bandwidth. However, existing CIM solutions require high implementation effort and lack flexibility from a software integration standpoint. This work proposes a novel, software-friendly, general-purpose, and low-integration-effort Near-Memory Computing (NMC) approach, paving the way for the adoption of CIM-based systems in the next generation of edge computing nodes. Two architectural variants, NM-Caesar and NM-Carus, are proposed and characterized to target different trade-offs in area efficiency, performance, and flexibility, covering a wide range of embedded microcontrollers. Post-layout simulations show up to $28.0\times$ and $53.9\times$ lower execution time and $25.0\times$ and $35.6\times$ higher energy efficiency at the system level, respectively, compared to executing the same tasks on a state-of-the-art RISC-V CPU (RV32IMC). NM-Carus achieves a peak energy efficiency of $306.7$ GOPS/W in 8-bit matrix multiplications, surpassing recent state-of-the-art in- and near-memory circuits.