Sleep-Based Homeostatic Regularization for Stabilizing Spike-Timing-Dependent Plasticity in Recurrent Spiking Neural Networks

TL;DR

This paper introduces a sleep-inspired regularization method for recurrent spiking neural networks that stabilizes learning by preventing weight saturation and preserving learned structures, inspired by biological sleep mechanisms.

Contribution

It proposes a novel sleep-based homeostatic regularization scheme for STDP in SNNs, demonstrating improved stability without hyperparameter tuning on benchmark datasets.

Findings

Sleep regularization prevents weight saturation in STDP-SNNs.

Improves stability and learning in MNIST-like benchmarks.

No benefit observed for surrogate-gradient SNNs with sleep protocol.

Abstract

Spike-timing-dependent plasticity (STDP) provides a biologically-plausible learning mechanism for spiking neural networks (SNNs); however, Hebbian weight updates in architectures with recurrent connections suffer from pathological weight dynamics: unbounded growth, catastrophic forgetting, and loss of representational diversity. We propose a neuromorphic regularization scheme inspired by the synaptic homeostasis hypothesis: periodic offline phases during which external inputs are suppressed, synaptic weights undergo stochastic decay toward a homeostatic baseline, and spontaneous activity enables memory consolidation. We demonstrate that this sleep-wake cycle prevents weight saturation while preserving learned structure. Empirically, we find that low to intermediate sleep durations (10-20\% of training) improve stability on MNIST-like benchmarks in our STDP-SNN model, without any data-specific hyperparameter tuning. In contrast, the same sleep intervention yields no measurable benefit for the surrogate-gradient spiking neural network (SG-SNN). Taken together, these results suggest that periodic, sleep-based renormalization may represent a fundamental mechanism for stabilizing local Hebbian learning in neuromorphic systems, while also indicating that special care is required when integrating such protocols with existing gradient-based optimization methods.