A Co-design view of Compute in-Memory with Non-Volatile Elements for Neural Networks

TL;DR

This paper explores the potential of non-volatile memory-based compute-in-memory architectures, particularly cross-bar designs, to overcome the von Neumann bottleneck in neural network processing, emphasizing co-design and material challenges.

Contribution

It provides a comprehensive co-design perspective on compute-in-memory architectures using non-volatile elements and reviews suitable materials and devices for future hardware development.

Findings

Cross-bar architectures enable efficient matrix-vector multiplication for neural networks.

Material properties and device challenges impact the performance and reliability of compute-in-memory systems.

Co-design approaches are essential for optimizing hardware and material integration.

Abstract

Deep Learning neural networks are pervasive, but traditional computer architectures are reaching the limits of being able to efficiently execute them for the large workloads of today. They are limited by the von Neumann bottleneck: the high cost in energy and latency incurred in moving data between memory and the compute engine. Today, special CMOS designs address this bottleneck. The next generation of computing hardware will need to eliminate or dramatically mitigate this bottleneck. We discuss how compute-in-memory can play an important part in this development. Here, a non-volatile memory based cross-bar architecture forms the heart of an engine that uses an analog process to parallelize the matrix vector multiplication operation, repeatedly used in all neural network workloads. The cross-bar architecture, at times referred to as a neuromorphic approach, can be a key hardware element in future computing machines. In the first part of this review we take a co-design view of the design constraints and the demands it places on the new materials and memory devices that anchor the cross-bar architecture. In the second part, we review what is knows about the different new non-volatile memory materials and devices suited for compute in-memory, and discuss the outlook and challenges.