Heterogeneous Neuronal and Synaptic Dynamics for Spike-Efficient Unsupervised Learning: Theory and Design Principles

TL;DR

This paper demonstrates that heterogeneity in neuronal and synaptic dynamics enhances the efficiency and performance of recurrent spiking neural networks by reducing spiking activity and increasing memory capacity, supported by theoretical analysis and empirical validation.

Contribution

It introduces a novel framework for designing heterogeneous RSNNs using Bayesian optimization to improve spike efficiency and prediction accuracy.

Findings

Heterogeneity reduces spiking activity while maintaining memory capacity.

Optimized HRSNN outperforms homogeneous RSNN in classification tasks.

Theoretical analysis links heterogeneity to improved learning and efficiency.

Abstract

This paper shows that the heterogeneity in neuronal and synaptic dynamics reduces the spiking activity of a Recurrent Spiking Neural Network (RSNN) while improving prediction performance, enabling spike-efficient (unsupervised) learning. We analytically show that the diversity in neurons' integration/relaxation dynamics improves an RSNN's ability to learn more distinct input patterns (higher memory capacity), leading to improved classification and prediction performance. We further prove that heterogeneous Spike-Timing-Dependent-Plasticity (STDP) dynamics of synapses reduce spiking activity but preserve memory capacity. The analytical results motivate Heterogeneous RSNN design using Bayesian optimization to determine heterogeneity in neurons and synapses to improve $\mathcal{E}$, defined as the ratio of spiking activity and memory capacity. The empirical results on time series classification and prediction tasks show that optimized HRSNN increases performance and reduces spiking activity compared to a homogeneous RSNN.