# Outstanding Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks

## TL;DR

This paper demonstrates that Binarized Neural Networks (BNNs) can tolerate high levels of bit errors in resistive RAM (RRAM) hardware, enabling more energy-efficient and robust in-memory neural network implementations.

## Contribution

It shows that BNNs can withstand significant bit errors in RRAM devices, and proposes training adaptations to enhance error tolerance, reducing hardware constraints.

## Key findings

- BNNs tolerate up to 10^-4 bit error rate with minimal performance loss.
- Training adaptation extends error tolerance to 4x10^-2 bit error rate.
- RRAM programming energy can be reduced by a factor of 30.

## Abstract

Resistive random access memories (RRAM) are novel nonvolatile memory technologies, which can be embedded at the core of CMOS, and which could be ideal for the in-memory implementation of deep neural networks. A particularly exciting vision is using them for implementing Binarized Neural Networks (BNNs), a class of deep neural networks with a highly reduced memory footprint. The challenge of resistive memory, however, is that they are prone to device variation, which can lead to bit errors. In this work we show that BNNs can tolerate these bit errors to an outstanding level, through simulations of networks on the MNIST and CIFAR10 tasks. If a standard BNN is used, up to 10^-4 bit error rate can be tolerated with little impact on recognition performance on both MNIST and CIFAR10. We then show that by adapting the training procedure to the fact that the BNN will be operated on error-prone hardware, this tolerance can be extended to a bit error rate of 4x10^-2. The requirements for RRAM are therefore a lot less stringent for BNNs than more traditional applications. We show, based on experimental measurements on a RRAM HfO2 technology, that this result can allow reduce RRAM programming energy by a factor 30.