Neuromorphic Acceleration for Approximate Bayesian Inference on Neural Networks via Permanent Dropout

TL;DR

This paper proposes a neuromorphic approach to perform approximate Bayesian inference using permanent dropout on spiking neural networks, reducing computational costs and enabling uncertainty quantification at scale or on edge devices.

Contribution

It demonstrates the feasibility of implementing permanent dropout-based uncertainty quantification on spiking neural networks using neuromorphic hardware, with minimal loss in predictive distribution accuracy.

Findings

Spiking neural networks can approximate classical Bayesian inference with permanent dropout.

The approach reduces computational and energy costs for uncertainty quantification.

Results on a drug therapy dataset show near-identical predictive distributions to classical networks.

Abstract

As neural networks have begun performing increasingly critical tasks for society, ranging from driving cars to identifying candidates for drug development, the value of their ability to perform uncertainty quantification (UQ) in their predictions has risen commensurately. Permanent dropout, a popular method for neural network UQ, involves injecting stochasticity into the inference phase of the model and creating many predictions for each of the test data. This shifts the computational and energy burden of deep neural networks from the training phase to the inference phase. Recent work has demonstrated near-lossless conversion of classical deep neural networks to their spiking counterparts. We use these results to demonstrate the feasibility of conducting the inference phase with permanent dropout on spiking neural networks, mitigating the technique's computational and energy burden, which is essential for its use at scale or on edge platforms. We demonstrate the proposed approach via the Nengo spiking neural simulator on a combination drug therapy dataset for cancer treatment, where UQ is critical. Our results indicate that the spiking approximation gives a predictive distribution practically indistinguishable from that given by the classical network.