Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks

TL;DR

This paper introduces a brain-inspired continual learning algorithm that reorganizes neural pathways in spiking neural networks to improve performance, energy efficiency, and memory capacity across diverse tasks.

Contribution

It proposes a novel adaptive reorganization method using Self-Organizing Regulation networks for efficient continual learning in spiking neural networks.

Findings

Outperforms existing models in diverse continual learning tasks.

Excels at learning complex tasks and integrating past knowledge.

Demonstrates self-repairing ability and automatic pathway allocation.

Abstract

The human brain can self-organize rich and diverse sparse neural pathways to incrementally master hundreds of cognitive tasks. However, most existing continual learning algorithms for deep artificial and spiking neural networks are unable to adequately auto-regulate the limited resources in the network, which leads to performance drop along with energy consumption rise as the increase of tasks. In this paper, we propose a brain-inspired continual learning algorithm with adaptive reorganization of neural pathways, which employs Self-Organizing Regulation networks to reorganize the single and limited Spiking Neural Network (SOR-SNN) into rich sparse neural pathways to efficiently cope with incremental tasks. The proposed model demonstrates consistent superiority in performance, energy consumption, and memory capacity on diverse continual learning tasks ranging from child-like simple to complex tasks, as well as on generalized CIFAR100 and ImageNet datasets. In particular, the SOR-SNN model excels at learning more complex tasks as well as more tasks, and is able to integrate the past learned knowledge with the information from the current task, showing the backward transfer ability to facilitate the old tasks. Meanwhile, the proposed model exhibits self-repairing ability to irreversible damage and for pruned networks, could automatically allocate new pathway from the retained network to recover memory for forgotten knowledge.