Rule Based Learning with Dynamic (Graph) Neural Networks

TL;DR

This paper introduces rule-based dynamic neural network layers that incorporate expert knowledge, enhancing graph neural networks' flexibility and performance by dynamically adapting parameters based on input data.

Contribution

The work presents a novel rule-based layer design for neural networks, especially applied to graph neural networks, allowing integration of expert knowledge and dynamic parameter arrangement.

Findings

RuleGNNs achieve comparable performance to state-of-the-art classifiers.

The approach generalizes classical neural network layers.

Synthetic datasets demonstrate improved integration of expert knowledge.

Abstract

A common problem of classical neural network architectures is that additional information or expert knowledge cannot be naturally integrated into the learning process. To overcome this limitation, we propose a two-step approach consisting of (1) generating rule functions from knowledge and (2) using these rules to define rule based layers -- a new type of dynamic neural network layer. The focus of this work is on the second step, i.e., rule based layers that are designed to dynamically arrange learnable parameters in the weight matrices and bias vectors depending on the input samples. Indeed, we prove that our approach generalizes classical feed-forward layers such as fully connected and convolutional layers by choosing appropriate rules. As a concrete application we present rule based graph neural networks (RuleGNNs) that overcome some limitations of ordinary graph neural networks. Our experiments show that the predictive performance of RuleGNNs is comparable to state-of-the-art graph classifiers using simple rules based on Weisfeiler-Leman labeling and pattern counting. Moreover, we introduce new synthetic benchmark graph datasets to show how to integrate expert knowledge into RuleGNNs making them more powerful than ordinary graph neural networks.