Scale up your In-Memory Accelerator: Leveraging Wireless-on-Chip Communication for AIMC-based CNN Inference

TL;DR

This paper proposes a large-scale AIMC architecture with wireless inter-tile communication to address high bandwidth and low latency data transfer challenges, enhancing CNN inference performance.

Contribution

It introduces a novel many-tile AIMC system leveraging wireless communication for scalable, high-performance CNN inference, integrating heterogeneous computing clusters.

Findings

Wireless communication significantly improves data transfer bandwidth.

Design space exploration shows potential for high efficiency gains.

Integration of RISC-V cores with AIMC tiles enhances flexibility.

Abstract

Analog In-Memory Computing (AIMC) is emerging as a disruptive paradigm for heterogeneous computing, potentially delivering orders of magnitude better peak performance and efficiency over traditional digital signal processing architectures on Matrix-Vector multiplication. However, to sustain this throughput in real-world applications, AIMC tiles must be supplied with data at very high bandwidth and low latency; this poses an unprecedented pressure on the on-chip communication infrastructure, which becomes the system's performance and efficiency bottleneck. In this context, the performance and plasticity of emerging on-chip wireless communication paradigms provide the required breakthrough to up-scale on-chip communication in large AIMC devices. This work presents a many-tile AIMC architecture with inter-tile wireless communication that integrates multiple heterogeneous computing clusters, embedding a mix of parallel RISC-V cores and AIMC tiles. We perform an extensive design space exploration of the proposed architecture and discuss the benefits of exploiting emerging on-chip communication technologies such as wireless transceivers in the millimeter-wave and terahertz bands.