PRUNIX: Non-Ideality Aware Convolutional Neural Network Pruning for Memristive Accelerators

TL;DR

PRUNIX introduces a novel framework for CNN pruning tailored for memristor-based accelerators, effectively handling non-ideal effects and maintaining high accuracy with significant sparsity.

Contribution

It proposes new regularization and pruning algorithms specifically designed to improve non-ideality tolerance and accuracy in memristor-based CNN deployment.

Findings

Achieved 13% higher test accuracy considering non-ideal effects.

Attained 85% sparsity in pruned CNNs.

Demonstrated improved robustness over standard methods.

Abstract

In this work, PRUNIX, a framework for training and pruning convolutional neural networks is proposed for deployment on memristor crossbar based accelerators. PRUNIX takes into account the numerous non-ideal effects of memristor crossbars including weight quantization, state-drift, aging and stuck-at-faults. PRUNIX utilises a novel Group Sawtooth Regularization intended to improve non-ideality tolerance as well as sparsity, and a novel Adaptive Pruning Algorithm (APA) intended to minimise accuracy loss by considering the sensitivity of different layers of a CNN to pruning. We compare our regularization and pruning methods with other standards on multiple CNN architectures, and observe an improvement of 13% test accuracy when quantization and other non-ideal effects are accounted for with an overall sparsity of 85%, which is similar to other methods