Memory Slices: A Modular Building Block for Scalable, Intelligent Memory Systems
Bahar Asgari, Saibal Mukhopadhyay, Sudhakar Yalamanchili

TL;DR
This paper introduces memory slices, a modular architecture that enhances scalable, high-performance neural network training by balancing bandwidth and compute, leading to superlinear speedup and improved power efficiency.
Contribution
It proposes a novel memory slice architecture that integrates compute and memory modularly, enabling scalable neural network training with balanced bandwidth and compute performance.
Findings
Memory slices achieve superlinear speedup with increasing slices.
Power efficiency reaches 747 GFLOPs/J for LSTM training.
The architecture is adaptable with various packaging options.
Abstract
While reduction in feature size makes computation cheaper in terms of latency, area, and power consumption, performance of emerging data-intensive applications is determined by data movement. These trends have introduced the concept of scalability as reaching a desirable performance per unit cost by using as few number of units as possible. Many proposals have moved compute closer to the memory. However, these efforts ignored maintaining a balance between bandwidth and compute rate of an architecture, with those of applications, which is a key principle in designing scalable large systems. This paper proposes the use of memory slices, a modular building block for scalable memory systems integrated with compute, in which performance scales with memory size (and volume of data). The slice architecture utilizes a programmable memory interface feeding a systolic compute engine with high…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Advanced Neural Network Applications · Cloud Computing and Resource Management
