Hemlet: A Heterogeneous Compute-in-Memory Chiplet Architecture for Vision Transformers with Group-Level Parallelism

TL;DR

Hemlet is a scalable, heterogeneous compute-in-memory chiplet architecture that accelerates Vision Transformers efficiently by employing group-level parallelism and system-level dataflow optimizations, achieving significant speedups and high energy efficiency.

Contribution

This work introduces Hemlet, a novel chiplet-based CIM system with group-level parallelism for scalable and efficient ViT acceleration, addressing communication and scalability challenges.

Findings

Achieves 2.41x to 5.74x speedup across configurations.

Reaches 9.56 TOPS throughput.

Energy efficiency of 4.98 TOPS/W.

Abstract

Vision Transformers (ViTs) have established new performance benchmarks in vision tasks such as image recognition and object detection. However, these advancements come with significant demands for memory and computational resources, presenting challenges for hardware deployment. Heterogeneous compute-in-memory (CIM) accelerators have emerged as a promising solution for enabling energy-efficient deployment of ViTs. Despite this potential, monolithic CIM-based designs face scalability issues due to the size limitations of a single chip. To address this challenge, emerging chiplet-based techniques offer a more scalable alternative. However, chiplet designs come with their own costs, as they introduce expensive communication, which can hinder improvements in throughput. This work introduces Hemlet, a heterogeneous CIM chiplet system designed to accelerate ViT workloads. Hemlet enables flexible resource scaling through the integration of heterogeneous analog CIM (ACIM), digital CIM (DCIM), and Intermediate Data Process (IDP) chiplets. To improve throughput while reducing communication overhead, it employs a group-level parallelism (GLP) mapping strategy and system-level dataflow optimization, achieving speedups ranging from 2.41x to 5.74x across various hardware configurations within the chiplet system. Our evaluation results show that Hemlet can reach a throughput of 9.56 TOPS with an energy efficiency of 4.98 TOPS/W.