Time Cell Inspired Temporal Codebook in Spiking Neural Networks for Enhanced Image Generation

TL;DR

This paper introduces a hippocampal-inspired temporal codebook in spiking neural networks to improve image generation, demonstrating state-of-the-art results across diverse datasets by capturing temporal dependencies.

Contribution

The paper proposes a novel temporal codebook mechanism inspired by hippocampal time cells for SNNs, enhancing generative performance on high-resolution and temporally complex datasets.

Findings

Outperforms existing SNN generative models on multiple datasets.

Excels in high-resolution and temporally consistent image generation.

Achieves state-of-the-art results on benchmark datasets.

Abstract

This paper presents a novel approach leveraging Spiking Neural Networks (SNNs) to construct a Variational Quantized Autoencoder (VQ-VAE) with a temporal codebook inspired by hippocampal time cells. This design captures and utilizes temporal dependencies, significantly enhancing the generative capabilities of SNNs. Neuroscientific research has identified hippocampal "time cells" that fire sequentially during temporally structured experiences. Our temporal codebook emulates this behavior by triggering the activation of time cell populations based on similarity measures as input stimuli pass through it. We conducted extensive experiments on standard benchmark datasets, including MNIST, FashionMNIST, CIFAR10, CelebA, and downsampled LSUN Bedroom, to validate our model's performance. Furthermore, we evaluated the effectiveness of the temporal codebook on neuromorphic datasets NMNIST and DVS-CIFAR10, and demonstrated the model's capability with high-resolution datasets such as CelebA-HQ, LSUN Bedroom, and LSUN Church. The experimental results indicate that our method consistently outperforms existing SNN-based generative models across multiple datasets, achieving state-of-the-art performance. Notably, our approach excels in generating high-resolution and temporally consistent data, underscoring the crucial role of temporal information in SNN-based generative modeling.