BSO: Binary Spiking Online Optimization Algorithm

TL;DR

This paper introduces BSO, an online training algorithm for Binary Spiking Neural Networks that reduces memory usage and improves training efficiency, with theoretical guarantees and superior experimental results.

Contribution

The paper presents BSO, a novel online training method for BSNNs that eliminates latent weights and introduces a temporal-aware variant, T-BSO, with convergence guarantees.

Findings

BSO reduces training memory significantly.

T-BSO captures temporal dynamics for better performance.

Both methods outperform existing training algorithms.

Abstract

Binary Spiking Neural Networks (BSNNs) offer promising efficiency advantages for resource-constrained computing. However, their training algorithms often require substantial memory overhead due to latent weights storage and temporal processing requirements. To address this issue, we propose Binary Spiking Online (BSO) optimization algorithm, a novel online training algorithm that significantly reduces training memory. BSO directly updates weights through flip signals under the online training framework. These signals are triggered when the product of gradient momentum and weights exceeds a threshold, eliminating the need for latent weights during training. To enhance performance, we propose T-BSO, a temporal-aware variant that leverages the inherent temporal dynamics of BSNNs by capturing gradient information across time steps for adaptive threshold adjustment. Theoretical analysis establishes convergence guarantees for both BSO and T-BSO, with formal regret bounds characterizing their convergence rates. Extensive experiments demonstrate that both BSO and T-BSO achieve superior optimization performance compared to existing training methods for BSNNs. The codes are available at https://github.com/hamings1/BSO.