Towards a Taxonomy of Graph Learning Datasets

TL;DR

This paper proposes a systematic taxonomy of graph datasets used in GNN benchmarking, revealing key data characteristics that influence model performance and guiding better evaluation and model development.

Contribution

It introduces a principled, data-driven approach to categorize graph datasets based on perturbations, enhancing understanding of what GNN models learn from different data aspects.

Findings

Identifies critical dataset characteristics affecting GNN performance

Provides a new framework for dataset evaluation and comparison

Facilitates development of more specialized GNN models

Abstract

Graph neural networks (GNNs) have attracted much attention due to their ability to leverage the intrinsic geometries of the underlying data. Although many different types of GNN models have been developed, with many benchmarking procedures to demonstrate the superiority of one GNN model over the others, there is a lack of systematic understanding of the underlying benchmarking datasets, and what aspects of the model are being tested. Here, we provide a principled approach to taxonomize graph benchmarking datasets by carefully designing a collection of graph perturbations to probe the essential data characteristics that GNN models leverage to perform predictions. Our data-driven taxonomization of graph datasets provides a new understanding of critical dataset characteristics that will enable better model evaluation and the development of more specialized GNN models.