StreamDCIM: A Tile-based Streaming Digital CIM Accelerator with Mixed-stationary Cross-forwarding Dataflow for Multimodal Transformer

TL;DR

StreamDCIM is a novel tile-based streaming digital CIM accelerator designed for multimodal Transformers, featuring reconfigurable microarchitecture, a mixed-stationary dataflow, and a pipelined compute-rewriting process to enhance efficiency and parallelism.

Contribution

It introduces a reconfigurable CIM macro architecture, a mixed-stationary dataflow, and a pipelined rewriting technique, addressing inflexibility in existing accelerators for multimodal Transformer processing.

Findings

Achieves 2.63× speedup over non-streaming CIM solutions.

Achieves 1.28× speedup over layer-based streaming CIM solutions.

Demonstrates improved utilization and parallelism in multimodal Transformer acceleration.

Abstract

Multimodal Transformers are emerging artificial intelligence (AI) models designed to process a mixture of signals from diverse modalities. Digital computing-in-memory (CIM) architectures are considered promising for achieving high efficiency while maintaining high accuracy. However, current digital CIM-based accelerators exhibit inflexibility in microarchitecture, dataflow, and pipeline to effectively accelerate multimodal Transformer. In this paper, we propose StreamDCIM, a tile-based streaming digital CIM accelerator for multimodal Transformers. It overcomes the above challenges with three features: First, we present a tile-based reconfigurable CIM macro microarchitecture with normal and hybrid reconfigurable modes to improve intra-macro CIM utilization. Second, we implement a mixed-stationary cross-forwarding dataflow with tile-based execution decoupling to exploit tile-level computation parallelism. Third, we introduce a ping-pong-like fine-grained compute-rewriting pipeline to overlap high-latency on-chip CIM rewriting. Experimental results show that StreamDCIM outperforms non-streaming and layer-based streaming CIM-based solutions by geomean 2.63$\times$ and 1.28$\times$ on typical multimodal Transformer models.