Dataflow Matrix Machines as a Generalization of Recurrent Neural Networks

TL;DR

Dataflow matrix machines extend recurrent neural networks by supporting multiple stream types, advanced neurons, and higher-order structures, promising broad applications in machine learning, probabilistic programming, and dynamic system synthesis.

Contribution

They introduce dataflow matrix machines as a versatile generalization of RNNs with support for diverse streams and higher-order constructs.

Findings

Support for multiple arbitrary stream types

Incorporation of powerful neuron models

Potential applications in machine learning and probabilistic programming

Abstract

Dataflow matrix machines are a powerful generalization of recurrent neural networks. They work with multiple types of arbitrary linear streams, multiple types of powerful neurons, and allow to incorporate higher-order constructions. We expect them to be useful in machine learning and probabilistic programming, and in the synthesis of dynamic systems and of deterministic and probabilistic programs.