Practical Bayesian Inference for Speech SNNs: Uncertainty and Loss-Landscape Smoothing

TL;DR

This paper introduces a Bayesian learning approach for Spiking Neural Networks in speech processing, demonstrating improved performance and smoother predictive landscapes on speech datasets.

Contribution

It applies Bayesian inference and the IVON method to SNNs, enhancing their predictive landscape smoothness and performance in speech tasks.

Findings

Improved negative log-likelihood and Brier score performance.

Smoother and more regular predictive landscape observed.

Effective application of Bayesian methods to SNNs for speech processing.

Abstract

Spiking Neural Networks (SNNs) are naturally suited for speech processing tasks due to their specific dynamics, which allows them to handle temporal data. However, the threshold-based generation of spikes in SNNs intuitively causes an angular or irregular predictive landscape. We explore the effect of using the Bayesian learning approach for the weights on the irregular predictive landscape. For the surrogate-gradient SNNs, we also explore the application of the Improved Variational Online Newton (IVON) approach, which is an efficient variational approach. The performance of the proposed approach is evaluated on the Heidelberg Digits and Speech Commands datasets. The hypothesis is that the Bayesian approach will result in a smoother and more regular predictive landscape, given the angular nature of the deterministic predictive landscape. The experimental evaluation of the proposed approach shows improved performance on the negative log-likelihood and Brier score. Furthermore, the proposed approach has resulted in a smoother and more regular predictive landscape compared to the deterministic approach, based on the one-dimensional slices of the weight space