Spiking Neural Networks Through the Lens of Streaming Algorithms

TL;DR

This paper explores the connection between streaming algorithms and spiking neural networks, designing neural algorithms inspired by streaming methods and establishing lower bounds through reductions, thus bridging computational models.

Contribution

It introduces a novel framework linking streaming algorithms with neural networks, providing both neural algorithms based on streaming techniques and lower bounds via reductions.

Findings

Neural algorithms for streaming tasks nearly match classical space bounds.

Efficient implementation of linear sketching in spiking neural networks.

Lower bounds in streaming translate to neural network space complexity bounds.

Abstract

We initiate the study of biological neural networks from the perspective of streaming algorithms. Like computers, human brains suffer from memory limitations which pose a significant obstacle when processing large scale and dynamically changing data. In computer science, these challenges are captured by the well-known streaming model, which can be traced back to Munro and Paterson `78 and has had significant impact in theory and beyond. In the classical streaming setting, one must compute some function $f$ of a stream of updates $\mathcal{S} = \{u_1,\ldots,u_m\}$, given restricted single-pass access to the stream. The primary complexity measure is the space used by the algorithm. We take the first steps towards understanding the connection between streaming and neural algorithms. On the upper bound side, we design neural algorithms based on known streaming algorithms for fundamental tasks, including distinct elements, approximate median, heavy hitters, and more. The number of neurons in our neural solutions almost matches the space bounds of the corresponding streaming algorithms. As a general algorithmic primitive, we show how to implement the important streaming technique of linear sketching efficient in spiking neural networks. On the lower bound side, we give a generic reduction, showing that any space-efficient spiking neural network can be simulated by a space-efficiently streaming algorithm. This reduction lets us translate streaming-space lower bounds into nearly matching neural-space lower bounds, establishing a close connection between these two models.