Spiking Neural Networks and Online Learning: An Overview and Perspectives

TL;DR

This paper provides a comprehensive overview of how Spiking Neural Networks can be effectively used for online learning in dynamic, non-stationary environments, highlighting their advantages over traditional models.

Contribution

It synthesizes current knowledge on Spiking Neural Networks for online learning and discusses their potential to adapt quickly to changing data distributions.

Findings

Spiking Neural Networks can adapt to data drift without retraining.

They outperform traditional models in online, non-stationary environments.

The paper encourages further research into SNNs for real-time learning applications.

Abstract

Applications that generate huge amounts of data in the form of fast streams are becoming increasingly prevalent, being therefore necessary to learn in an online manner. These conditions usually impose memory and processing time restrictions, and they often turn into evolving environments where a change may affect the input data distribution. Such a change causes that predictive models trained over these stream data become obsolete and do not adapt suitably to new distributions. Specially in these non-stationary scenarios, there is a pressing need for new algorithms that adapt to these changes as fast as possible, while maintaining good performance scores. Unfortunately, most off-the-shelf classification models need to be retrained if they are used in changing environments, and fail to scale properly. Spiking Neural Networks have revealed themselves as one of the most successful approaches to model the behavior and learning potential of the brain, and exploit them to undertake practical online learning tasks. Besides, some specific flavors of Spiking Neural Networks can overcome the necessity of retraining after a drift occurs. This work intends to merge both fields by serving as a comprehensive overview, motivating further developments that embrace Spiking Neural Networks for online learning scenarios, and being a friendly entry point for non-experts.