Overcoming Order in Autoregressive Graph Generation

TL;DR

This paper introduces an Orderless Regularization (OLR) for RNN-based graph generation, making models invariant to node ordering and improving performance, especially with limited data.

Contribution

It proposes a novel regularization technique that addresses the order sensitivity in sequential graph generation models, enhancing their robustness and diversity.

Findings

Regularization improves graph generation quality.

Model performs better with limited data.

Invariance to node ordering is achieved.

Abstract

Graph generation is a fundamental problem in various domains, including chemistry and social networks. Recent work has shown that molecular graph generation using recurrent neural networks (RNNs) is advantageous compared to traditional generative approaches which require converting continuous latent representations into graphs. One issue which arises when treating graph generation as sequential generation is the arbitrary order of the sequence which results from a particular choice of graph flattening method. In this work we propose using RNNs, taking into account the non-sequential nature of graphs by adding an Orderless Regularization (OLR) term that encourages the hidden state of the recurrent model to be invariant to different valid orderings present under the training distribution. We demonstrate that sequential graph generation models benefit from our proposed regularization scheme, especially when data is scarce. Our findings contribute to the growing body of research on graph generation and provide a valuable tool for various applications requiring the synthesis of realistic and diverse graph structures.