SatSOM: Saturation Self-Organizing Maps for Continual Learning

TL;DR

SatSOM introduces a saturation mechanism to self-organizing maps, enhancing their ability to retain knowledge during continual learning by freezing well-trained neurons and focusing learning on underutilized areas.

Contribution

The paper presents SatSOM, a novel extension of SOMs that improves continual learning by reducing forgetting through a saturation-based neuron management strategy.

Findings

SatSOM reduces catastrophic forgetting in continual learning tasks.

SatSOM maintains higher accuracy over sequential tasks compared to standard SOMs.

The saturation mechanism effectively preserves learned knowledge while allowing adaptation.

Abstract

Continual learning poses a fundamental challenge for neural systems, which often suffer from catastrophic forgetting when exposed to sequential tasks. Self-Organizing Maps (SOMs), despite their interpretability and efficiency, are not immune to this issue. In this paper, we introduce Saturation Self-Organizing Maps (SatSOM)-an extension of SOMs designed to improve knowledge retention in continual learning scenarios. SatSOM incorporates a novel saturation mechanism that gradually reduces the learning rate and neighborhood radius of neurons as they accumulate information. This effectively freezes well-trained neurons and redirects learning to underutilized areas of the map.