Revisiting Graph Homophily Measures

TL;DR

This paper introduces an unbiased homophily measure for graphs that overcomes limitations of existing measures, enabling reliable comparison across datasets with different class distributions, and discusses the challenges in defining such measures for directed graphs.

Contribution

The paper proposes a new homophily measure with all desirable properties, addressing limitations of existing measures and analyzing the theoretical constraints for directed graphs.

Findings

Existing homophily measures have significant drawbacks.

Unbiased homophily satisfies all desirable properties for undirected graphs.

No measure can satisfy all properties for directed graphs due to inherent contradictions.

Abstract

Homophily is a graph property describing the tendency of edges to connect similar nodes. There are several measures used for assessing homophily but all are known to have certain drawbacks: in particular, they cannot be reliably used for comparing datasets with varying numbers of classes and class size balance. To show this, previous works on graph homophily suggested several properties desirable for a good homophily measure, also noting that no existing homophily measure has all these properties. Our paper addresses this issue by introducing a new homophily measure - unbiased homophily - that has all the desirable properties and thus can be reliably used across datasets with different label distributions. The proposed measure is suitable for undirected (and possibly weighted) graphs. We show both theoretically and via empirical examples that the existing homophily measures have serious drawbacks while unbiased homophily has a desirable behavior for the considered scenarios. Finally, when it comes to directed graphs, we prove that some desirable properties contradict each other and thus a measure satisfying all of them cannot exist.