Dataflow Matrix Machines and V-values: a Bridge between Programs and Neural Nets

TL;DR

This paper introduces Dataflow Matrix Machines (DMMs), a generalized neural network framework supporting flexible input/output arities, self-referential mechanisms, and unbounded growth, bridging neural nets and programming.

Contribution

It presents DMMs as a versatile extension of neural networks with dynamic, self-referential capabilities and introduces V-values for flexible data representation.

Findings

DMMs support arbitrary input/output arities and unbounded network growth.

V-values enable variadic activation functions and data structure representation.

DMMs facilitate general-purpose programming with neural network principles.

Abstract

1) Dataflow matrix machines (DMMs) generalize neural nets by replacing streams of numbers with linear streams (streams supporting linear combinations), allowing arbitrary input and output arities for activation functions, countable-sized networks with finite dynamically changeable active part capable of unbounded growth, and a very expressive self-referential mechanism. 2) DMMs are suitable for general-purpose programming, while retaining the key property of recurrent neural networks: programs are expressed via matrices of real numbers, and continuous changes to those matrices produce arbitrarily small variations in the associated programs. 3) Spaces of V-values (vector-like elements based on nested maps) are particularly useful, enabling DMMs with variadic activation functions and conveniently representing conventional data structures.