ALADE-SNN: Adaptive Logit Alignment in Dynamically Expandable Spiking Neural Networks for Class Incremental Learning

TL;DR

This paper introduces ALADE-SNN, a novel adaptive logit alignment framework for spiking neural networks that enhances class incremental learning by balancing feature representation and managing old task weights, achieving state-of-the-art results.

Contribution

The paper proposes the ALADE-SNN framework with adaptive logit alignment and OtoN suppression, improving continual learning in SNNs and dynamically adjusting network architecture based on task demands.

Findings

Achieves 75.42% accuracy on CIFAR100-B0 benchmark over 10 steps.

Surpasses existing SNN-based continual learning methods.

Balances performance between new and old tasks effectively.

Abstract

Inspired by the human brain's ability to adapt to new tasks without erasing prior knowledge, we develop spiking neural networks (SNNs) with dynamic structures for Class Incremental Learning (CIL). Our comparative experiments reveal that limited datasets introduce biases in logits distributions among tasks. Fixed features from frozen past-task extractors can cause overfitting and hinder the learning of new tasks. To address these challenges, we propose the ALADE-SNN framework, which includes adaptive logit alignment for balanced feature representation and OtoN suppression to manage weights mapping frozen old features to new classes during training, releasing them during fine-tuning. This approach dynamically adjusts the network architecture based on analytical observations, improving feature extraction and balancing performance between new and old tasks. Experiment results show that ALADE-SNN achieves an average incremental accuracy of 75.42 on the CIFAR100-B0 benchmark over 10 incremental steps. ALADE-SNN not only matches the performance of DNN-based methods but also surpasses state-of-the-art SNN-based continual learning algorithms. This advancement enhances continual learning in neuromorphic computing, offering a brain-inspired, energy-efficient solution for real-time data processing.