Continual Competitive Memory: A Neural System for Online Task-Free Lifelong Learning

TL;DR

This paper introduces continual competitive memory (CCM), a novel unsupervised neural system that effectively mitigates catastrophic forgetting in online lifelong learning, demonstrating competitive performance on standard benchmarks.

Contribution

The paper presents CCM, a new neural model based on competition principles, unifying existing models and improving lifelong learning performance without task supervision.

Findings

CCM outperforms other competitive neural models.

CCM achieves results comparable to state-of-the-art lifelong learning methods.

CCM effectively reduces catastrophic forgetting in online classification tasks.

Abstract

In this article, we propose a novel form of unsupervised learning, continual competitive memory (CCM), as well as a computational framework to unify related neural models that operate under the principles of competition. The resulting neural system is shown to offer an effective approach for combating catastrophic forgetting in online continual classification problems. We demonstrate that the proposed CCM system not only outperforms other competitive learning neural models but also yields performance that is competitive with several modern, state-of-the-art lifelong learning approaches on benchmarks such as Split MNIST and Split NotMNIST. CCM yields a promising path forward for acquiring representations that are robust to interference from data streams, especially when the task is unknown to the model and must be inferred without external guidance.