STT-RAM-based Hierarchical In-Memory Computing

TL;DR

This paper investigates hierarchical in-memory computing using STT-RAM, analyzing tradeoffs between processing in memory and cache, and demonstrating advantages of STT-RAM-based cache architectures for certain workloads.

Contribution

It introduces a hierarchical in-memory computing framework utilizing non-volatile and relaxed-retention STT-RAM, analyzing performance tradeoffs and proposing heterogeneous cache architectures.

Findings

STT-RAM-based PiC outperforms PiM for specific workloads.

Tradeoffs between data movement and write overheads are workload-dependent.

Heterogeneous STT-RAM cache architectures can optimize performance.

Abstract

In-memory computing promises to overcome the von Neumann bottleneck in computer systems by performing computations directly within the memory. Previous research has suggested using Spin-Transfer Torque RAM (STT-RAM) for in-memory computing due to its non-volatility, low leakage power, high density, endurance, and commercial viability. This paper explores hierarchical in-memory computing, where different levels of the memory hierarchy are augmented with processing elements to optimize workload execution. The paper investigates processing in memory (PiM) using non-volatile STT-RAM and processing in cache (PiC) using volatile STT-RAM with relaxed retention, which helps mitigate STT-RAM's write latency and energy overheads. We analyze tradeoffs and overheads associated with data movement for PiC versus write overheads for PiM using STT-RAMs for various workloads. We examine workload characteristics, such as computational intensity and CPU-dependent workloads with limited instruction-level parallelism, and their impact on PiC/PiM tradeoffs. Using these workloads, we evaluate computing in STT-RAM versus SRAM at different cache hierarchy levels and explore the potential of heterogeneous STT-RAM cache architectures with various retention times for PiC and CPU-based computing. Our experiments reveal significant advantages of STT-RAM-based PiC over PiM for specific workloads. Finally, we describe open research problems in hierarchical in-memory computing architectures to further enhance this paradigm.