CIMple: Standard-cell SRAM-based CIM with LUT-based split softmax for attention acceleration

TL;DR

This paper introduces CIMple, a digital SRAM-based compute-in-memory accelerator for self-attention in transformer models, achieving high efficiency and low latency suitable for edge deployment of large language models.

Contribution

It proposes a novel dual-banked CIM architecture with LUT-based fixed-point implementation for efficient self-attention acceleration in transformers.

Findings

Achieves 26.1 TOPS/W at 0.85V in 28nm CMOS.

Attains 2.31 TOPS/mm$^2$ at 1.2V with INT8 precision.

Demonstrates suitability for resource-constrained edge devices.

Abstract

Large Language Models (LLMs) such as LLaMA and DeepSeek, are built on transformer architectures, which have become a standard model for achieving state-of-the-art performance in natural language processing tasks. Recently, there has been growing interest in deploying LLMs on edge devices. Although smaller LLM models are being proposed, they often still contain billions of parameters. Since edge devices are limited in their resources this poses a significant challenge for edge deployment. Compute-in-memory (CIM) is a promising architecture that addresses this by reducing data movement through the integration of computational logic directly into memory. However, existing CIM architectures support only static Multiply-Accumulate (MAC) operations which limit their configurability in supporting nonlinear operations and various types of transformer models. This paper presents a fully digital standard-cell SRAM-based CIM architecture accelerator for self-attention, called CIMple, designed to overcome these limitations, inside transformer models. The key contributions of CIMple are: 1) A novel dual-banked CIM-based fully digital self-attention accelerator using 8-bit parallel weight feeding. 2) A look-up-table (LUT) based fixed-point implementation reducing latency with minimal accuracy degradation. 3) A performance evaluation of a 32kb CIM-based self-attention accelerator implemented in 28nm, which achieves 26.1 TOPS/W at 0.85V and 2.31 TOPS/mm$^2$ at 1.2V, both with INT8 precision.