Transfer between long-term and short-term memory using Conceptors

TL;DR

This paper presents a neural network model that integrates long-term and short-term memory using conceptors and gated reservoirs, enabling bidirectional information transfer and manipulation of memories.

Contribution

It introduces a novel model combining conceptors with gated reservoirs to unify long-term and short-term memory in neural networks.

Findings

Model effectively transfers information between memory types

Standard conceptor operations manipulate stored memories

Demonstrates bidirectional memory flow in neural network

Abstract

We introduce a recurrent neural network model of working memory combining short-term and long-term components. e short-term component is modelled using a gated reservoir model that is trained to hold a value from an input stream when a gate signal is on. e long-term component is modelled using conceptors in order to store inner temporal patterns (that corresponds to values). We combine these two components to obtain a model where information can go from long-term memory to short-term memory and vice-versa and we show how standard operations on conceptors allow to combine long-term memories and describe their effect on short-term memory.