Relational dynamic memory networks

TL;DR

The paper introduces Relational Dynamic Memory Networks (RMDN), a neural architecture with structured, graph-based external memory designed to better handle complex data structures and answer queries without explicit programming.

Contribution

It presents a novel MANN with multi-relational graph memory, enabling neural networks to manipulate structured data for various prediction tasks.

Findings

RMDN effectively predicts software vulnerabilities.

RMDN accurately models molecular bioactivity.

RMDN successfully predicts chemical interactions.

Abstract

Neural networks excel in detecting regular patterns but are less successful in representing and manipulating complex data structures, possibly due to the lack of an external memory. This has led to the recent development of a new line of architectures known as Memory-Augmented Neural Networks (MANNs), each of which consists of a neural network that interacts with an external memory matrix. However, this RAM-like memory matrix is unstructured and thus does not naturally encode structured objects. Here we design a new MANN dubbed Relational Dynamic Memory Network (RMDN) to bridge the gap. Like existing MANNs, RMDN has a neural controller but its memory is structured as multi-relational graphs. RMDN uses the memory to represent and manipulate graph-structured data in response to query; and as a neural network, RMDN is trainable from labeled data. Thus RMDN learns to answer queries about a set of graph-structured objects without explicit programming. We evaluate the capability of RMDN on several important prediction problems, including software vulnerability, molecular bioactivity and chemical-chemical interaction. Results demonstrate the efficacy of the proposed model.