A Hybrid Spiking-Convolutional Neural Network Approach for Advancing Machine Learning Models

TL;DR

This paper introduces a hybrid Spiking-Convolutional Neural Network that combines event-based spiking neurons with traditional CNNs to improve image inpainting, demonstrating state-of-the-art results on a custom dataset.

Contribution

The paper presents a novel standalone hybrid SC-NN model integrating SNN and CNN layers for image inpainting, leveraging temporal processing and spatial feature learning.

Findings

Achieved a training loss of 0.015 and validation loss of 0.0017.

Demonstrated state-of-the-art performance on image inpainting tasks.

Validated the effectiveness of combining spiking and convolutional layers.

Abstract

In this article, we propose a novel standalone hybrid Spiking-Convolutional Neural Network (SC-NN) model and test on using image inpainting tasks. Our approach uses the unique capabilities of SNNs, such as event-based computation and temporal processing, along with the strong representation learning abilities of CNNs, to generate high-quality inpainted images. The model is trained on a custom dataset specifically designed for image inpainting, where missing regions are created using masks. The hybrid model consists of SNNConv2d layers and traditional CNN layers. The SNNConv2d layers implement the leaky integrate-and-fire (LIF) neuron model, capturing spiking behavior, while the CNN layers capture spatial features. In this study, a mean squared error (MSE) loss function demonstrates the training process, where a training loss value of 0.015, indicates accurate performance on the training set and the model achieved a validation loss value as low as 0.0017 on the testing set. Furthermore, extensive experimental results demonstrate state-of-the-art performance, showcasing the potential of integrating temporal dynamics and feature extraction in a single network for image inpainting.