Dataflow matrix machines as programmable, dynamically expandable, self-referential generalized recurrent neural networks

TL;DR

Dataflow matrix machines extend recurrent neural networks by supporting multiple stream types and higher-order neurons, enabling dynamic self-modification and serving as a versatile programming platform.

Contribution

This paper introduces dataflow matrix machines as a flexible, self-referential neural network framework with dynamic topology and weight updates, expanding the capabilities of traditional RNNs.

Findings

Supports multiple stream and neuron types

Enables dynamic network topology updates

Provides useful programming idioms for neural computation

Abstract

Dataflow matrix machines are a powerful generalization of recurrent neural networks. They work with multiple types of linear streams and multiple types of neurons, including higher-order neurons which dynamically update the matrix describing weights and topology of the network in question while the network is running. It seems that the power of dataflow matrix machines is sufficient for them to be a convenient general purpose programming platform. This paper explores a number of useful programming idioms and constructions arising in this context.