Notes on Pure Dataflow Matrix Machines: Programming with Self-referential Matrix Transformations

TL;DR

This paper introduces Pure Dataflow Matrix Machines, a programming paradigm for self-referential neural networks using streams of matrices, aiming to establish a lambda-calculus-like foundation for this architecture.

Contribution

It proposes a novel programming discipline for dataflow matrix machines based solely on streams of matrices, enabling self-referential and pure dataflow network definitions.

Findings

Defines a programming framework with one stream type: matrices.

Shows how to construct self-referential neural networks within this framework.

Provides a foundation for a lambda-calculus analogue in matrix-based neural programming.

Abstract

Dataflow matrix machines are self-referential generalized recurrent neural nets. The self-referential mechanism is provided via a stream of matrices defining the connectivity and weights of the network in question. A natural question is: what should play the role of untyped lambda-calculus for this programming architecture? The proposed answer is a discipline of programming with only one kind of streams, namely the streams of appropriately shaped matrices. This yields Pure Dataflow Matrix Machines which are networks of transformers of streams of matrices capable of defining a pure dataflow matrix machine.