GRAN is superior to GraphRNN: node orderings, kernel- and graph embeddings-based metrics for graph generators

TL;DR

This paper compares graph generative models using advanced metrics, demonstrating GRAN's superiority over GraphRNN, especially with optimized node orderings, and provides practical guidelines and open-source tools for future research.

Contribution

The study introduces a comprehensive evaluation framework for graph generators using manifold-based metrics and reveals the impact of node orderings, establishing GRAN's advantages over GraphRNN.

Findings

Manifold-based metrics outperform kernel-based metrics in embedding space.

GRAN outperforms GraphRNN in generating graphs.

Node ordering significantly affects model performance.

Abstract

A wide variety of generative models for graphs have been proposed. They are used in drug discovery, road networks, neural architecture search, and program synthesis. Generating graphs has theoretical challenges, such as isomorphic representations -- evaluating how well a generative model performs is difficult. Which model to choose depending on the application domain? We extensively study kernel-based metrics on distributions of graph invariants and manifold-based and kernel-based metrics in graph embedding space. Manifold-based metrics outperform kernel-based metrics in embedding space. We use these metrics to compare GraphRNN and GRAN, two well-known generative models for graphs, and unveil the influence of node orderings. It shows the superiority of GRAN over GraphRNN - further, our proposed adaptation of GraphRNN with a depth-first search ordering is effective for small-sized graphs. A guideline on good practices regarding dataset selection and node feature initialization is provided. Our work is accompanied by open-source code and reproducible experiments.