Learning in Memristive Neural Network Architectures using Analog Backpropagation Circuits

TL;DR

This paper presents the design and verification of analog backpropagation circuits for memristive neural networks, enabling on-chip learning for various architectures like DNN, BNN, MNN, HTM, and LSTM, validated through simulations.

Contribution

It introduces novel analog backpropagation circuits tailored for memristive architectures, addressing the open challenge of circuit-level implementation of learning algorithms.

Findings

Successful circuit design and verification using CMOS and memristor models

Application validation on XOR, MNIST, and Yale face datasets

Demonstrated feasibility of on-chip learning in memristive neural networks

Abstract

The on-chip implementation of learning algorithms would speed-up the training of neural networks in crossbar arrays. The circuit level design and implementation of backpropagation algorithm using gradient descent operation for neural network architectures is an open problem. In this paper, we proposed the analog backpropagation learning circuits for various memristive learning architectures, such as Deep Neural Network (DNN), Binary Neural Network (BNN), Multiple Neural Network (MNN), Hierarchical Temporal Memory (HTM) and Long-Short Term Memory (LSTM). The circuit design and verification is done using TSMC 180nm CMOS process models, and TiO2 based memristor models. The application level validations of the system are done using XOR problem, MNIST character and Yale face image databases