Training Dynamic Exponential Family Models with Causal and Lateral Dependencies for Generalized Neuromorphic Computing

TL;DR

This paper introduces a probabilistic model for spiking neural networks with causal and lateral dependencies, extending exponential family models to time series, and provides distributed learning rules for training.

Contribution

It presents a novel generalized exponential family model for SNNs with causal and lateral dependencies, along with distributed learning algorithms for maximum likelihood and Bayesian training.

Findings

Model captures complex dependencies in SNNs

Distributed learning rules enable scalable training

Extends exponential family models to time series context

Abstract

Neuromorphic hardware platforms, such as Intel's Loihi chip, support the implementation of Spiking Neural Networks (SNNs) as an energy-efficient alternative to Artificial Neural Networks (ANNs). SNNs are networks of neurons with internal analogue dynamics that communicate by means of binary time series. In this work, a probabilistic model is introduced for a generalized set-up in which the synaptic time series can take values in an arbitrary alphabet and are characterized by both causal and instantaneous statistical dependencies. The model, which can be considered as an extension of exponential family harmoniums to time series, is introduced by means of a hybrid directed-undirected graphical representation. Furthermore, distributed learning rules are derived for Maximum Likelihood and Bayesian criteria under the assumption of fully observed time series in the training set.