Towards Efficient In-memory Computing Hardware for Quantized Neural Networks: State-of-the-art, Open Challenges and Perspectives

TL;DR

This paper reviews the current state and challenges of implementing quantized neural networks on in-memory computing hardware, emphasizing energy efficiency and hardware-software integration for edge AI.

Contribution

It offers a comprehensive overview of IMC-based QNNs, linking software quantization techniques to hardware design, and discusses open challenges and future perspectives.

Findings

Highlights the importance of quantization for energy-efficient edge AI

Identifies key challenges in IMC hardware implementation of QNNs

Provides a roadmap for future IMC-based QNN hardware development

Abstract

The amount of data processed in the cloud, the development of Internet-of-Things (IoT) applications, and growing data privacy concerns force the transition from cloud-based to edge-based processing. Limited energy and computational resources on edge push the transition from traditional von Neumann architectures to In-memory Computing (IMC), especially for machine learning and neural network applications. Network compression techniques are applied to implement a neural network on limited hardware resources. Quantization is one of the most efficient network compression techniques allowing to reduce the memory footprint, latency, and energy consumption. This paper provides a comprehensive review of IMC-based Quantized Neural Networks (QNN) and links software-based quantization approaches to IMC hardware implementation. Moreover, open challenges, QNN design requirements, recommendations, and perspectives along with an IMC-based QNN hardware roadmap are provided.