HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapses

TL;DR

HetSyn introduces synaptic heterogeneity with diverse time constants into spiking neural networks, enhancing their temporal processing, robustness, and generalization, inspired by biological neurons.

Contribution

The paper presents HetSyn, a novel framework modeling synaptic heterogeneity with specific time constants, improving SNN performance and biological plausibility.

Findings

Improves SNN performance across multiple tasks.

Enhances robustness to noise and limited resources.

Aligns learned synaptic timescales with biological data.

Abstract

Spiking Neural Networks (SNNs) offer a biologically plausible and energy-efficient framework for temporal information processing. However, existing studies overlook a fundamental property widely observed in biological neurons-synaptic heterogeneity, which plays a crucial role in temporal processing and cognitive capabilities. To bridge this gap, we introduce HetSyn, a generalized framework that models synaptic heterogeneity with synapse-specific time constants. This design shifts temporal integration from the membrane potential to the synaptic current, enabling versatile timescale integration and allowing the model to capture diverse synaptic dynamics. We implement HetSyn as HetSynLIF, an extended form of the leaky integrate-and-fire (LIF) model equipped with synapse-specific decay dynamics. By adjusting the parameter configuration, HetSynLIF can be specialized into vanilla LIF neurons, neurons with threshold adaptation, and neuron-level heterogeneous models. We demonstrate that HetSynLIF not only improves the performance of SNNs across a variety of tasks-including pattern generation, delayed match-to-sample, speech recognition, and visual recognition-but also exhibits strong robustness to noise, enhanced working memory performance, efficiency under limited neuron resources, and generalization across timescales. In addition, analysis of the learned synaptic time constants reveals trends consistent with empirical observations in biological synapses. These findings underscore the significance of synaptic heterogeneity in enabling efficient neural computation, offering new insights into brain-inspired temporal modeling.