A data-centric approach for assessing progress of Graph Neural Networks

TL;DR

This paper introduces a data-centric approach for evaluating multi-label graph neural networks, including new datasets, metrics, and a comprehensive comparison of existing methods.

Contribution

It provides the first multi-label graph datasets, new homophily metrics for multi-label scenarios, and a large-scale comparative study of GNN methods.

Findings

New multi-label biological datasets released

Proposed homophily metrics for multi-label graphs

Comparative analysis of 8 methods across 9 datasets

Abstract

Graph Neural Networks (GNNs) have achieved state-of-the-art results in node classification tasks. However, most improvements are in multi-class classification, with less focus on the cases where each node could have multiple labels. The first challenge in studying multi-label node classification is the scarcity of publicly available datasets. To address this, we collected and released three real-world biological datasets and developed a multi-label graph generator with tunable properties. We also argue that traditional notions of homophily and heterophily do not apply well to multi-label scenarios. Therefore, we define homophily and Cross-Class Neighborhood Similarity for multi-label classification and investigate $9$ collected multi-label datasets. Lastly, we conducted a large-scale comparative study with $8$ methods across nine datasets to evaluate current progress in multi-label node classification. We release our code at \url{https://github.com/Tianqi-py/MLGNC}.