Gradient-based Bit Encoding Optimization for Noise-Robust Binary Memristive Crossbar

TL;DR

This paper introduces GBO, a gradient-based method to optimize binary bit encoding in memristive crossbars, significantly improving noise robustness and accuracy without extensive noise data collection.

Contribution

It proposes a novel layer-wise bit encoding optimization approach that enhances noise robustness in memristive crossbars, addressing limitations of previous noise-aware training methods.

Findings

GBO improves classification accuracy by 5-40% under severe noise.

Layer-wise bit encoding adapts to different noise sensitivities across layers.

The method achieves high noise robustness with low computational cost.

Abstract

Binary memristive crossbars have gained huge attention as an energy-efficient deep learning hardware accelerator. Nonetheless, they suffer from various noises due to the analog nature of the crossbars. To overcome such limitations, most previous works train weight parameters with noise data obtained from a crossbar. These methods are, however, ineffective because it is difficult to collect noise data in large-volume manufacturing environment where each crossbar has a large device/circuit level variation. Moreover, we argue that there is still room for improvement even though these methods somewhat improve accuracy. This paper explores a new perspective on mitigating crossbar noise in a more generalized way by manipulating input binary bit encoding rather than training the weight of networks with respect to noise data. We first mathematically show that the noise decreases as the number of binary bit encoding pulses increases when representing the same amount of information. In addition, we propose Gradient-based Bit Encoding Optimization (GBO) which optimizes a different number of pulses at each layer, based on our in-depth analysis that each layer has a different level of noise sensitivity. The proposed heterogeneous layer-wise bit encoding scheme achieves high noise robustness with low computational cost. Our experimental results on public benchmark datasets show that GBO improves the classification accuracy by ~5-40% in severe noise scenarios.