End-to-End DNN Inference on a Massively Parallel Analog In Memory Computing Architecture

TL;DR

This paper demonstrates end-to-end inference of ResNet-18 on a large-scale analog in-memory computing architecture, achieving high performance and providing insights for future many-core AIMC systems.

Contribution

It presents the first full inference of ResNet-18 on a 512-cluster AIMC-RISC-V architecture, showcasing scalability and performance improvements.

Findings

Achieved up to 20.2 TOPS performance.

Mapped ResNet-18 effectively on non-volatile memory cells.

Provided design guidelines for future AIMC many-core architectures.

Abstract

The demand for computation resources and energy efficiency of Convolutional Neural Networks (CNN) applications requires a new paradigm to overcome the "Memory Wall". Analog In-Memory Computing (AIMC) is a promising paradigm since it performs matrix-vector multiplications, the critical kernel of many ML applications, in-place in the analog domain within memory arrays structured as crossbars of memory cells. However, several factors limit the full exploitation of this technology, including the physical fabrication of the crossbar devices, which constrain the memory capacity of a single array. Multi-AIMC architectures have been proposed to overcome this limitation, but they have been demonstrated only for tiny and custom CNNs or performing some layers off-chip. In this work, we present the full inference of an end-to-end ResNet-18 DNN on a 512-cluster heterogeneous architecture coupling a mix of AIMC cores and digital RISC-V cores, achieving up to 20.2 TOPS. Moreover, we analyze the mapping of the network on the available non-volatile cells, compare it with state-of-the-art models, and derive guidelines for next-generation many-core architectures based on AIMC devices.