Exploiting Oxide Based Resistive RAM Variability for Bayesian Neural Network Hardware Design

TL;DR

This paper proposes a hardware design that leverages the inherent variability of oxide-based RRAMs to implement probabilistic sampling for Bayesian neural networks, enhancing uncertainty modeling in machine learning.

Contribution

It introduces a novel approach to utilize RRAM variability for Bayesian sampling, bridging hardware variability with probabilistic neural network requirements.

Findings

Demonstrates RRAM variability can be harnessed for probabilistic sampling.

Provides a hardware implementation for Bayesian neural networks.

Shows potential for improved uncertainty estimation in ML hardware.

Abstract

Uncertainty plays a key role in real-time machine learning. As a significant shift from standard deep networks, which does not consider any uncertainty formulation during its training or inference, Bayesian deep networks are being currently investigated where the network is envisaged as an ensemble of plausible models learnt by the Bayes' formulation in response to uncertainties in sensory data. Bayesian deep networks consider each synaptic weight as a sample drawn from a probability distribution with learnt mean and variance. This paper elaborates on a hardware design that exploits cycle-to-cycle variability of oxide based Resistive Random Access Memories (RRAMs) as a means to realize such a probabilistic sampling function, instead of viewing it as a disadvantage.