An 818-TOPS/W CSNR-31dB SQNR-45dB 10-bit Capacitor-Reconfiguring Computing-in-Memory Macro with Software-Analog Co-Design for Transformers

TL;DR

This paper introduces a high-accuracy, energy-efficient analog CIM macro with capacitor reconfiguration and software-analog co-design, enabling transformer inference with state-of-the-art performance and power efficiency.

Contribution

It proposes a novel capacitor-reconfiguring CIM with integrated software-analog co-design, achieving high accuracy and efficiency for transformer inference.

Findings

Achieves 818 TOPS/W power efficiency.

Attains 95.8% CIFAR10 accuracy with analog CIMs.

Improves FoM by 2.3x in SQNR and 1.5x in CSNR.

Abstract

Transformer inference requires high compute accuracy; achieving this using analog CIMs has been difficult due to inherent computational errors. To overcome this challenge, we propose a Capacitor-Reconfiguring CIM (CR-CIM) to realize high compute accuracy analog CIM with a 10-bit ADC attaining high-area/power efficiency. CR-CIM reconfigures its capacitor array to serve dual purposes: for computation and ADC conversion, achieving significant area savings. Furthermore, CR-CIMs eliminate signal attenuation by keeping the signal charge stationary during operation, leading to a 4x improvement in comparator energy efficiency. We also propose a software-analog co-design technique integrating majority voting into the 10-bit ADC to dynamically optimize the CIM noise performance based on the running layer to further save inference power. Our CR-CIM achieves the highest compute-accuracy for analog CIMs, and the power efficiency of 818 TOPS/W is competitive with the state-of-the-art. Furthermore, the FoM considering SQNR and CSNR is 2.3x and 1.5x better than previous works, respectively. Vision Transformer (ViT) inference is achieved and realizes a highest CIFAR10 accuracy of 95.8% for analog CIMs.