NeuroTrainer: An Intelligent Memory Module for Deep Learning Training

TL;DR

NeuroTrainer introduces an intelligent, energy-efficient memory module with in-memory accelerators and a programmable data flow model, significantly improving deep neural network training performance across diverse architectures.

Contribution

The paper proposes a novel scalable architecture integrating in-memory accelerators with a flexible data flow model for efficient DNN training.

Findings

Achieves 500 GFLOPS/W power efficiency in simulations.

Supports diverse DNN architectures with high throughput.

Demonstrates energy-efficient training in 15nm FinFET design.

Abstract

This paper presents, NeuroTrainer, an intelligent memory module with in-memory accelerators that forms the building block of a scalable architecture for energy efficient training for deep neural networks. The proposed architecture is based on integration of a homogeneous computing substrate composed of multiple processing engines in the logic layer of a 3D memory module. NeuroTrainer utilizes a programmable data flow based execution model to optimize memory mapping and data re-use during different phases of training operation. A programming model and supporting architecture utilizes the flexible data flow to efficiently accelerate training of various types of DNNs. The cycle level simulation and synthesized design in 15nm FinFET showspower efficiency of 500 GFLOPS/W, and almost similar throughput for a wide range of DNNs including convolutional, recurrent, multi-layer-perceptron, and mixed (CNN+RNN) networks