Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era

TL;DR

This paper explores how leveraging Dark Memory and specialized accelerators in dark silicon systems can optimize energy efficiency and performance through co-design of algorithms and hardware, using Pareto analysis.

Contribution

It introduces Pareto curves for energy and area metrics to optimize accelerator and memory design in dark silicon systems, emphasizing the importance of Dark Memory.

Findings

Memory access limits energy efficiency per operation.

Dark Memory is essential for high-performance systems.

Co-design of algorithms and hardware is crucial for optimization.

Abstract

The key challenge to improving performance in the age of Dark Silicon is how to leverage transistors when they cannot all be used at the same time. In modern SOCs, these transistors are often used to create specialized accelerators which improve energy efficiency for some applications by 10-1000X. While this might seem like the magic bullet we need, for most CPU applications more energy is dissipated in the memory system than in the processor: these large gains in efficiency are only possible if the DRAM and memory hierarchy are mostly idle. We refer to this desirable state as Dark Memory, and it only occurs for applications with an extreme form of locality. To show our findings, we introduce Pareto curves in the energy/op and mm$^2$/(ops/s) metric space for compute units, accelerators, and on-chip memory/interconnect. These Pareto curves allow us to solve the power, performance, area constrained optimization problem to determine which accelerators should be used, and how to set their design parameters to optimize the system. This analysis shows that memory accesses create a floor to the achievable energy-per-op. Thus high performance requires Dark Memory, which in turn requires co-design of the algorithm for parallelism and locality, with the hardware.