Use it or Lose it: Selective Memory and Forgetting in a Perpetual Learning Machine

TL;DR

This paper introduces a Perpetual Learning Machine (PLM) that mimics human-like memory by selectively retaining or forgetting information based on recall frequency during continuous learning.

Contribution

It demonstrates how a neural network can simulate human-like selective memory and forgetting through biased recall during perpetual stochastic gradient descent.

Findings

Frequent recall leads to memory retention.

Rarely recalled memories are forgotten.

Memory process is stimulus-driven and similar to human memory.

Abstract

In a recent article we described a new type of deep neural network - a Perpetual Learning Machine (PLM) - which is capable of learning 'on the fly' like a brain by existing in a state of Perpetual Stochastic Gradient Descent (PSGD). Here, by simulating the process of practice, we demonstrate both selective memory and selective forgetting when we introduce statistical recall biases during PSGD. Frequently recalled memories are remembered, whilst memories recalled rarely are forgotten. This results in a 'use it or lose it' stimulus driven memory process that is similar to human memory.