Spiking Manifesto

TL;DR

This paper advocates for a paradigm shift to spiking neural networks (SNNs) inspired by the brain's efficiency, proposing a new architecture with vastly larger encoding capacity and lower energy consumption compared to traditional AI models.

Contribution

It introduces a novel framework for interpreting AI models through the lens of spiking activity and polychronization, aiming to develop more efficient neural architectures.

Findings

SNNs have factorially larger encoding capacity than ANNs.

Spiking activity can be viewed as nature's look-up tables.

Potential for a thousandfold performance improvement.

Abstract

Practically everything computers do is better, faster, and more power-efficient than the brain. For example, a calculator performs numerical computations more energy-efficiently than any human. Yet modern AI models are a thousand times less efficient than the brain. These models rely on larger and larger artificial neural networks (ANNs) to boost their encoding capacity, requiring GPUs to perform large-scale matrix multiplications. In contrast, the brain's spiking neural networks (SNNs) exhibit factorially explosive encoding capacity and compute through the polychronization of spikes rather than explicit matrix-vector products, resulting in lower energy requirements. This manifesto proposes a paradigm for framing popular AI models in terms of spiking networks and polychronization, and for interpreting spiking activity as nature's way of implementing look-up tables. This suggests a path toward converting AI models into a novel class of architectures with much smaller size yet combinatorially large encoding capacity, offering the promise of a thousandfold improvement in performance. Code is available at https://github.com/izhikevich/SNN