Examining and Mitigating the Impact of Crossbar Non-idealities for Accurate Implementation of Sparse Deep Neural Networks

TL;DR

This paper investigates how crossbar non-idealities affect the accuracy of sparse deep neural networks and proposes mitigation techniques to improve their performance on non-ideal crossbar hardware.

Contribution

It provides a comprehensive analysis of non-idealities' impact on sparse DNNs and introduces two mitigation methods, column rearrangement and weight-constrained training.

Findings

Highly sparse DNNs suffer significant accuracy loss on non-ideal crossbars.

Mitigation techniques improve accuracy by increasing low conductance synapses.

Structured pruning alone does not account for hardware non-idealities.

Abstract

Recently several structured pruning techniques have been introduced for energy-efficient implementation of Deep Neural Networks (DNNs) with lesser number of crossbars. Although, these techniques have claimed to preserve the accuracy of the sparse DNNs on crossbars, none have studied the impact of the inexorable crossbar non-idealities on the actual performance of the pruned networks. To this end, we perform a comprehensive study to show how highly sparse DNNs, that result in significant crossbar-compression-rate, can lead to severe accuracy losses compared to unpruned DNNs mapped onto non-ideal crossbars. We perform experiments with multiple structured-pruning approaches (such as, C/F pruning, XCS and XRS) on VGG11 and VGG16 DNNs with benchmark datasets (CIFAR10 and CIFAR100). We propose two mitigation approaches - Crossbar column rearrangement and Weight-Constrained-Training (WCT) - that can be integrated with the crossbar-mapping of the sparse DNNs to minimize accuracy losses incurred by the pruned models. These help in mitigating non-idealities by increasing the proportion of low conductance synapses on crossbars, thereby improving their computational accuracies.