Spiking representation learning for associative memories

TL;DR

This paper introduces a novel spiking neural network model inspired by neocortical organization that learns representations and forms associative memories, addressing scalability and real-world data challenges in SNNs.

Contribution

The work presents a new unsupervised SNN architecture combining Hebbian plasticity and structural plasticity for associative memory and representation learning.

Findings

Effective pattern completion and perceptual rivalry demonstrated

Resistant to distortions in memory retrieval

Capable of extracting prototypes from data

Abstract

Networks of interconnected neurons communicating through spiking signals offer the bedrock of neural computations. Our brains spiking neural networks have the computational capacity to achieve complex pattern recognition and cognitive functions effortlessly. However, solving real-world problems with artificial spiking neural networks (SNNs) has proved to be difficult for a variety of reasons. Crucially, scaling SNNs to large networks and processing large-scale real-world datasets have been challenging, especially when compared to their non-spiking deep learning counterparts. The critical operation that is needed of SNNs is the ability to learn distributed representations from data and use these representations for perceptual, cognitive and memory operations. In this work, we introduce a novel SNN that performs unsupervised representation learning and associative memory operations leveraging Hebbian synaptic and activity-dependent structural plasticity coupled with neuron-units modelled as Poisson spike generators with sparse firing (~1 Hz mean and ~100 Hz maximum firing rate). Crucially, the architecture of our model derives from the neocortical columnar organization and combines feedforward projections for learning hidden representations and recurrent projections for forming associative memories. We evaluated the model on properties relevant for attractor-based associative memories such as pattern completion, perceptual rivalry, distortion resistance, and prototype extraction.