Technology Aware Training in Memristive Neuromorphic Systems based on non-ideal Synaptic Crossbars

TL;DR

This paper addresses the impact of non-idealities in memristive crossbar implementations on neural network accuracy and proposes a technology aware training method to mitigate these effects, enabling larger crossbar sizes with minimal accuracy loss.

Contribution

It introduces a mathematical model of non-idealities in memristive crossbars and a training algorithm that compensates for these non-idealities to preserve neural network performance.

Findings

Non-idealities can cause up to 62% accuracy degradation.

The proposed training method recovers accuracy within 1.9% of ideal.

Enables larger crossbar arrays with minimal accuracy trade-offs.

Abstract

The advances in the field of machine learning using neuromorphic systems have paved the pathway for extensive research on possibilities of hardware implementations of neural networks. Various memristive technologies such as oxide-based devices, spintronics and phase change materials have been explored to implement the core functional units of neuromorphic systems, namely the synaptic network, and the neuronal functionality, in a fast and energy efficient manner. However, various non-idealities in the crossbar implementations of the synaptic arrays can significantly degrade performance of neural networks and hence, impose restrictions on feasible crossbar sizes. In this work, we build mathematical models of various non-idealities that occur in crossbar implementations such as source resistance, neuron resistance and chip-to-chip device variations and analyze their impact on the classification accuracy of a fully connected network (FCN) and convolutional neural network (CNN) trained with standard training algorithm. We show that a network trained under ideal conditions can suffer accuracy degradation as large as 59.84% for FCNs and 62.4% for CNNs when implemented on non-ideal crossbars for relevant non-ideality ranges. This severely constrains the sizes for crossbars. As a solution, we propose a technology aware training algorithm which incorporates the mathematical models of the non-idealities in the standard training algorithm. We demonstrate that our proposed methodology achieves significant recovery of testing accuracy within 1.9% of the ideal accuracy for FCNs and 1.5% for CNNs. We further show that our proposed training algorithm can potentially allow the use of significantly larger crossbar arrays of sizes 784$\times$500 for FCNs and 4096$\times$512 for CNNs with a minor or no trade-off in accuracy