Programming Patterns in Dataflow Matrix Machines and Generalized Recurrent Neural Nets

TL;DR

This paper explores programming patterns in dataflow matrix machines, a powerful generalization of recurrent neural networks, highlighting their expressive capabilities and potential as a general-purpose programming platform.

Contribution

It introduces the concept of programming patterns in dataflow matrix machines and analyzes their connectivity structures, expanding understanding of their expressive power.

Findings

Dataflow matrix machines can be synthesized from matrices of numbers.

They have sufficient expressive power for general-purpose programming.

Connectivity patterns correspond to programming structures.

Abstract

Dataflow matrix machines arise naturally in the context of synchronous dataflow programming with linear streams. They can be viewed as a rather powerful generalization of recurrent neural networks. Similarly to recurrent neural networks, large classes of dataflow matrix machines are described by matrices of numbers, and therefore dataflow matrix machines can be synthesized by computing their matrices. At the same time, the evidence is fairly strong that dataflow matrix machines have sufficient expressive power to be a convenient general-purpose programming platform. Because of the network nature of this platform, programming patterns often correspond to patterns of connectivity in the generalized recurrent neural networks understood as programs. This paper explores a variety of such programming patterns.