Neural Turing Machines

TL;DR

Neural Turing Machines integrate neural networks with external memory, enabling the learning of simple algorithms through differentiable operations, representing a significant step towards neural systems with algorithmic capabilities.

Contribution

This paper introduces Neural Turing Machines, a novel neural architecture that couples neural networks with external memory for differentiable algorithm learning.

Findings

Successfully learned copying, sorting, and associative recall algorithms

Demonstrated end-to-end differentiable training of neural memory systems

Showed potential for neural systems to perform algorithmic tasks

Abstract

We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.