Folding over Neural Networks

TL;DR

This paper proposes a novel recursive data type approach in Haskell for representing and training neural networks, emphasizing structure and semantics separation for improved compositionality.

Contribution

It introduces a recursive data type encoding for neural networks and applies recursion schemes for training, enhancing modularity and clarity.

Findings

Recursive representation enables structured traversal of networks.

Separation of structure and semantics improves compositionality.

Recursion schemes facilitate training processes.

Abstract

Neural networks are typically represented as data structures that are traversed either through iteration or by manual chaining of method calls. However, a deeper analysis reveals that structured recursion can be used instead, so that traversal is directed by the structure of the network itself. This paper shows how such an approach can be realised in Haskell, by encoding neural networks as recursive data types, and then their training as recursion scheme patterns. In turn, we promote a coherent implementation of neural networks that delineates between their structure and semantics, allowing for compositionality in both how they are built and how they are trained.