Design Space Exploration of Dense and Sparse Mapping Schemes for RRAM Architectures

TL;DR

This paper introduces a comprehensive design space exploration methodology to compare dense and sparse mapping schemes in RRAM architectures, analyzing their performance, power efficiency, and noise susceptibility across various neural network configurations.

Contribution

It extends existing DSE methods to evaluate the trade-offs of dense versus sparse mappings in RRAM-based neural accelerators, considering non-idealities and different network architectures.

Findings

Sparse mappings reduce power consumption but are more noise-sensitive.

Dense mappings offer better robustness against device non-idealities.

Trade-offs depend on network architecture and tile size.

Abstract

The impact of device and circuit-level effects in mixed-signal Resistive Random Access Memory (RRAM) accelerators typically manifest as performance degradation of Deep Learning (DL) algorithms, but the degree of impact varies based on algorithmic features. These include network architecture, capacity, weight distribution, and the type of inter-layer connections. Techniques are continuously emerging to efficiently train sparse neural networks, which may have activation sparsity, quantization, and memristive noise. In this paper, we present an extended Design Space Exploration (DSE) methodology to quantify the benefits and limitations of dense and sparse mapping schemes for a variety of network architectures. While sparsity of connectivity promotes less power consumption and is often optimized for extracting localized features, its performance on tiled RRAM arrays may be more susceptible to noise due to under-parameterization, when compared to dense mapping schemes. Moreover, we present a case study quantifying and formalizing the trade-offs of typical non-idealities introduced into 1-Transistor-1-Resistor (1T1R) tiled memristive architectures and the size of modular crossbar tiles using the CIFAR-10 dataset.