Hardware-aware Training Techniques for Improving Robustness of Ex-Situ Neural Network Transfer onto Passive TiO2 ReRAM Crossbars

TL;DR

This paper introduces hardware-aware training techniques tailored for TiO2 ReRAM crossbars to enhance neural network robustness during transfer, demonstrating significant accuracy improvements over traditional training methods.

Contribution

The work proposes novel training approaches that incorporate hardware variability considerations into neural network training for ReRAM crossbars, improving transfer accuracy.

Findings

79.5% of test points classified with ≥95% accuracy using hardware-aware training

Only 18.5% of test points achieve ≥95% accuracy with regular training

Hardware-aware models outperform regular models in transfer robustness

Abstract

Passive resistive random access memory (ReRAM) crossbar arrays, a promising emerging technology used for analog matrix-vector multiplications, are far superior to their active (1T1R) counterparts in terms of the integration density. However, current transfers of neural network weights into the conductance state of the memory devices in the crossbar architecture are accompanied by significant losses in precision due to hardware variabilities such as sneak path currents, biasing scheme effects and conductance tuning imprecision. In this work, training approaches that adapt techniques such as dropout, the reparametrization trick and regularization to TiO2 crossbar variabilities are proposed in order to generate models that are better adapted to their hardware transfers. The viability of this approach is demonstrated by comparing the outputs and precision of the proposed hardware-aware network with those of a regular fully connected network over a few thousand weight transfers using the half moons dataset in a simulation based on experimental data. For the neural network trained using the proposed hardware-aware method, 79.5% of the test set's data points can be classified with an accuracy of 95% or higher, while only 18.5% of the test set's data points can be classified with this accuracy by the regularly trained neural network.