The Role of Isomorphism Classes in Multi-Relational Datasets

TL;DR

This paper investigates how awareness of isomorphism classes in multi-relational datasets impacts neural relational inference models, revealing biases, proposing benchmarks, and improving model performance and training stability.

Contribution

It introduces isomorphism-aware synthetic benchmarks and demonstrates how leveraging isomorphism classes enhances model generalisation and training efficiency.

Findings

Isomorphism leakage inflates performance estimates.

Sampling biases impair model generalisation.

Isomorphism-aware sampling improves training stability.

Abstract

Multi-interaction systems abound in nature, from colloidal suspensions to gene regulatory circuits. These systems can produce complex dynamics and graph neural networks have been proposed as a method to extract underlying interactions and predict how systems will evolve. The current training and evaluation procedures for these models through the use of synthetic multi-relational datasets however are agnostic to interaction network isomorphism classes, which produce identical dynamics up to initial conditions. We extensively analyse how isomorphism class awareness affects these models, focusing on neural relational inference (NRI) models, which are unique in explicitly inferring interactions to predict dynamics in the unsupervised setting. Specifically, we demonstrate that isomorphism leakage overestimates performance in multi-relational inference and that sampling biases present in the multi-interaction network generation process can impair generalisation. To remedy this, we propose isomorphism-aware synthetic benchmarks for model evaluation. We use these benchmarks to test generalisation abilities and demonstrate the existence of a threshold sampling frequency of isomorphism classes for successful learning. In addition, we demonstrate that isomorphism classes can be utilised through a simple prioritisation scheme to improve model performance, stability during training and reduce training time.