Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay

TL;DR

This paper introduces the continual SOM (c-SOM), a novel approach that reduces catastrophic forgetting in self-organizing maps by leveraging internally-induced generative replay, enhancing lifelong learning capabilities.

Contribution

The study proposes c-SOM, a new model that mitigates forgetting in SOMs during incremental learning through internally-generated data replay mechanisms.

Findings

c-SOM significantly reduces forgetting in incremental learning tasks.

Internally-induced generative replay improves memory retention in SOMs.

c-SOM outperforms traditional SOMs in continual learning scenarios.

Abstract

A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt in this way is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day. While forgetting in the context of feedforward networks has been examined extensively over the decades, far less has been done in the context of alternative architectures such as the venerable self-organizing map (SOM), an unsupervised neural model that is often used in tasks such as clustering and dimensionality reduction. Although the competition among its internal neurons might carry the potential to improve memory retention, we observe that a fixed-sized SOM trained on task incremental data, i.e., it receives data points related to specific classes at certain temporal increments, experiences significant forgetting. In this study, we propose the continual SOM (c-SOM), a model that is capable of reducing its own forgetting when processing information.