An Introduction to Probabilistic Spiking Neural Networks: Probabilistic Models, Learning Rules, and Applications

TL;DR

This paper introduces probabilistic models and learning rules for spiking neural networks, emphasizing their energy efficiency and potential for direct training, with a focus on variational inference-based methods.

Contribution

It presents a probabilistic signal processing framework for SNNs, deriving new learning rules directly from first principles, advancing beyond existing biologically plausible or conversion-based methods.

Findings

Derivation of supervised and unsupervised learning rules from variational inference.

Highlighting the energy efficiency advantages of SNNs with event-driven processing.

Identification of open research problems in probabilistic SNNs.

Abstract

Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms lags behind the hardware implementations. Most existing training algorithms for SNNs have been designed either for biological plausibility or through conversion from pretrained ANNs via rate encoding. This article provides an introduction to SNNs by focusing on a probabilistic signal processing methodology that enables the direct derivation of learning rules by leveraging the unique time-encoding capabilities of SNNs. We adopt discrete-time probabilistic models for networked spiking neurons and derive supervised and unsupervised learning rules from first principles via variational inference. Examples and open research problems are also provided.