Similarity-based context aware continual learning for spiking neural networks

TL;DR

This paper introduces SCA-SNN, a novel continual learning algorithm for spiking neural networks that leverages task similarity to improve knowledge reuse, reduce energy consumption, and enhance biological plausibility.

Contribution

The paper proposes a similarity-based context-aware continual learning method for SNNs that adaptively reuses and expands neurons based on task similarity, improving efficiency and performance.

Findings

Outperforms existing SNN and DNN continual learning methods on multiple datasets.

Reduces energy consumption through selective neuron reuse.

Enhances biological interpretability of continual learning models.

Abstract

Biological brains have the capability to adaptively coordinate relevant neuronal populations based on the task context to learn continuously changing tasks in real-world environments. However, existing spiking neural network-based continual learning algorithms treat each task equally, ignoring the guiding role of different task similarity associations for network learning, which limits knowledge utilization efficiency. Inspired by the context-dependent plasticity mechanism of the brain, we propose a Similarity-based Context Aware Spiking Neural Network (SCA-SNN) continual learning algorithm to efficiently accomplish task incremental learning and class incremental learning. Based on contextual similarity across tasks, the SCA-SNN model can adaptively reuse neurons from previous tasks that are beneficial for new tasks (the more similar, the more neurons are reused) and flexibly expand new neurons for the new task (the more similar, the fewer neurons are expanded). Selective reuse and discriminative expansion significantly improve the utilization of previous knowledge and reduce energy consumption. Extensive experimental results on CIFAR100, ImageNet generalized datasets, and FMNIST-MNIST, SVHN-CIFAR100 mixed datasets show that our SCA-SNN model achieves superior performance compared to both SNN-based and DNN-based continual learning algorithms. Additionally, our algorithm has the capability to adaptively select similar groups of neurons for related tasks, offering a promising approach to enhancing the biological interpretability of efficient continual learning.