Vector Quantized Bayesian Neural Network Inference for Data Streams

TL;DR

This paper introduces VQ-BNN, a novel approach that approximates Bayesian neural network inference for data streams, significantly reducing computational costs while maintaining high-quality uncertainty estimation.

Contribution

VQ-BNN is a new model that predicts once and uses temporal smoothing to approximate Bayesian inference efficiently for data streams.

Findings

VQ-BNN performs faster than traditional BNNs.

VQ-BNN achieves comparable or better predictive accuracy.

VQ-BNN maintains uncertainty estimation quality.

Abstract

Bayesian neural networks (BNN) can estimate the uncertainty in predictions, as opposed to non-Bayesian neural networks (NNs). However, BNNs have been far less widely used than non-Bayesian NNs in practice since they need iterative NN executions to predict a result for one data, and it gives rise to prohibitive computational cost. This computational burden is a critical problem when processing data streams with low-latency. To address this problem, we propose a novel model VQ-BNN, which approximates BNN inference for data streams. In order to reduce the computational burden, VQ-BNN inference predicts NN only once and compensates the result with previously memorized predictions. To be specific, VQ-BNN inference for data streams is given by temporal exponential smoothing of recent predictions. The computational cost of this model is almost the same as that of non-Bayesian NNs. Experiments including semantic segmentation on real-world data show that this model performs significantly faster than BNNs while estimating predictive results comparable to or superior to the results of BNNs.