# Stochastic Computing for Hardware Implementation of Binarized Neural Networks

## TL;DR

This paper introduces a stochastic computing approach for Binarized Neural Networks, enabling fully binarized inputs and significant hardware savings in area and energy, with minimal accuracy loss.

## Contribution

It proposes a stochastic computing method for Binarized Neural Networks with binarized inputs and demonstrates hardware efficiency gains through ASIC implementation.

## Key findings

- Approaching performance of conventional Binarized Neural Networks on datasets.
- 62% area reduction in ASIC implementation.
- Energy savings of over 50% with slight accuracy reduction.

## Abstract

Binarized Neural Networks, a recently discovered class of neural networks with minimal memory requirements and no reliance on multiplication, are a fantastic opportunity for the realization of compact and energy efficient inference hardware. However, such neural networks are generally not entirely binarized: their first layer remains with fixed point input. In this work, we propose a stochastic computing version of Binarized Neural Networks, where the input is also binarized. Simulations on the example of the Fashion-MNIST and CIFAR-10 datasets show that such networks can approach the performance of conventional Binarized Neural Networks. We evidence that the training procedure should be adapted for use with stochastic computing. Finally, the ASIC implementation of our scheme is investigated, in a system that closely associates logic and memory, implemented by Spin Torque Magnetoresistive Random Access Memory. This analysis shows that the stochastic computing approach can allow considerable savings with regards to conventional Binarized Neural networks in terms of area (62% area reduction on the Fashion-MNIST task). It can also allow important savings in terms of energy consumption, if we accept reasonable reduction of accuracy: for example a factor 2.1 can be saved, with the cost of 1.4% in Fashion-MNIST test accuracy. These results highlight the high potential of Binarized Neural Networks for hardware implementation, and that adapting them to hardware constrains can provide important benefits.