Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyond

TL;DR

This paper critically examines measures of homophily in graphs, introduces a more robust measure called adjusted homophily, and proposes label informativeness (LI) to better characterize heterophily and predict GNN performance.

Contribution

It formalizes desirable properties for homophily measures, introduces adjusted homophily and LI, and demonstrates LI's effectiveness in correlating with GNN performance.

Findings

Adjusted homophily satisfies more desirable properties than other measures.

LI better correlates with GNN performance than traditional homophily measures.

Proposes a new framework for characterizing heterophily beyond the simple dichotomy.

Abstract

Homophily is a graph property describing the tendency of edges to connect similar nodes; the opposite is called heterophily. It is often believed that heterophilous graphs are challenging for standard message-passing graph neural networks (GNNs), and much effort has been put into developing efficient methods for this setting. However, there is no universally agreed-upon measure of homophily in the literature. In this work, we show that commonly used homophily measures have critical drawbacks preventing the comparison of homophily levels across different datasets. For this, we formalize desirable properties for a proper homophily measure and verify which measures satisfy which properties. In particular, we show that a measure that we call adjusted homophily satisfies more desirable properties than other popular homophily measures while being rarely used in graph machine learning literature. Then, we go beyond the homophily-heterophily dichotomy and propose a new characteristic that allows one to further distinguish different sorts of heterophily. The proposed label informativeness (LI) characterizes how much information a neighbor's label provides about a node's label. We prove that this measure satisfies important desirable properties. We also observe empirically that LI better agrees with GNN performance compared to homophily measures, which confirms that it is a useful characteristic of the graph structure.