GEM3D CIM General Purpose Matrix Computation Using 3D Integrated SRAM eDRAM Hybrid Compute In Memory on Memory Architecture

TL;DR

This paper introduces a 3D-integrated SRAM-eDRAM hybrid CIM architecture capable of efficiently performing general matrix operations directly within memory, extending CIM beyond dot products for versatile AI and computing applications.

Contribution

The work presents a novel 3D memory-on-memory CIM architecture supporting general matrix operations, including transpose and element-wise calculations, with optimized design for latency, energy, and density.

Findings

Supports general matrix operations with 4-bit precision.

Balances latency, energy efficiency, and compute density.

Enables broader AI acceleration and high-performance computing applications.

Abstract

With the rapid growth of deep neural networks (DNNs), compute-in-memory (CIM) has emerged as a promising energy-efficient paradigm for accelerating multiply-and-accumulate (MAC) operations. Yet, current CIM architectures are largely limited to dot-product computations and struggle to efficiently support general-purpose matrix operations, such as transpose, element-wise addition, and multiplication. This work presents a 3D-integrated, memory-on-memory SRAM-eDRAM hybrid CIM architecture, implemented in GlobalFoundries 22~nm FDSOI technology, capable of performing general matrix operations directly within the memory crossbar with 4-bit precision. By leveraging a specialized transpose-based architecture, in-memory arithmetic operations, peripheral-aware design, and 3D SRAM--eDRAM integration, the proposed architecture balances latency, energy efficiency, and compute density for general purpose matrix operations while remaining compatible with the conventional CIM dot product architectures. Overall, this memory-on-memory CIM framework generalizes CIM beyond dot products, enabling versatile matrix processing and paving the way for broader applications in AI acceleration and general-purpose high performance computing.