An associative memory for the on-line recognition and prediction of temporal sequences

TL;DR

This paper introduces an associative memory model capable of on-line learning and prediction of temporal sequences, combining neural and shift register functionalities for scalable, context-sensitive sequence processing.

Contribution

It proposes a novel framework and model for on-line sequence learning using an associative memory with a context store and sparse distributed memory.

Findings

Model can store and predict various sequence types and lengths

Sensitivity to sequence context is controllable

Numerical simulations demonstrate the model's properties

Abstract

This paper presents the design of an associative memory with feedback that is capable of on-line temporal sequence learning. A framework for on-line sequence learning has been proposed, and different sequence learning models have been analysed according to this framework. The network model is an associative memory with a separate store for the sequence context of a symbol. A sparse distributed memory is used to gain scalability. The context store combines the functionality of a neural layer with a shift register. The sensitivity of the machine to the sequence context is controllable, resulting in different characteristic behaviours. The model can store and predict on-line sequences of various types and length. Numerical simulations on the model have been carried out to determine its properties.