Reservoir memory machines

TL;DR

Reservoir memory machines extend echo state networks with external memory, enabling long-term storage and solving benchmark tasks efficiently, while being easier to train than Neural Turing Machines.

Contribution

The paper introduces reservoir memory machines, a new model combining echo state networks with external memory, simplifying training and enhancing long-term storage capabilities.

Findings

Can solve benchmark tests similar to Neural Turing Machines

Require only an alignment algorithm and linear regression for training

Enable arbitrarily long storage without interference

Abstract

In recent years, Neural Turing Machines have gathered attention by joining the flexibility of neural networks with the computational capabilities of Turing machines. However, Neural Turing Machines are notoriously hard to train, which limits their applicability. We propose reservoir memory machines, which are still able to solve some of the benchmark tests for Neural Turing Machines, but are much faster to train, requiring only an alignment algorithm and linear regression. Our model can also be seen as an extension of echo state networks with an external memory, enabling arbitrarily long storage without interference.