A Neural Turing~Machine for Conditional Transition Graph Modeling

TL;DR

This paper introduces the Conditional Neural Turing Machine (CNTM), an extension of NTM that models conditional transition graphs influenced by external context, demonstrated on synthetic and real-world data with high accuracy.

Contribution

The paper presents a novel extension of the Neural Turing Machine that learns conditional transition graphs influenced by external information, addressing complex cyclic graph modeling.

Findings

Achieved up to 82.12% accuracy on 10-node graphs.

Achieved up to 65.25% accuracy on 100-node graphs.

Successfully modeled transition paths in crisis-related information retrieval graphs.

Abstract

Graphs are an essential part of many machine learning problems such as analysis of parse trees, social networks, knowledge graphs, transportation systems, and molecular structures. Applying machine learning in these areas typically involves learning the graph structure and the relationship between the nodes of the graph. However, learning the graph structure is often complex, particularly when the graph is cyclic, and the transitions from one node to another are conditioned such as graphs used to represent a finite state machine. To solve this problem, we propose to extend the memory based Neural Turing Machine (NTM) with two novel additions. We allow for transitions between nodes to be influenced by information received from external environments, and we let the NTM learn the context of those transitions. We refer to this extension as the Conditional Neural Turing Machine (CNTM). We show that the CNTM can infer conditional transition graphs by empirically verifiying the model on two data sets: a large set of randomly generated graphs, and a graph modeling the information retrieval process during certain crisis situations. The results show that the CNTM is able to reproduce the paths inside the graph with accuracy ranging from 82,12% for 10 nodes graphs to 65,25% for 100 nodes graphs.