Extended Graph Assessment Metrics for Graph Neural Networks

TL;DR

This paper introduces extended graph assessment metrics for graph neural networks, applicable to regression tasks and continuous adjacency matrices, improving evaluation of graph structures in medical applications.

Contribution

It extends existing graph assessment metrics to cover regression tasks and continuous adjacency matrices, including new definitions for homophily and CCNS, and demonstrates their correlation with GNN performance.

Findings

Extended GAMs for regression and continuous graphs.

Correlation between new metrics and GNN performance.

Improved evaluation of graph structures in medical datasets.

Abstract

When re-structuring patient cohorts into so-called population graphs, initially independent data points can be incorporated into one interconnected graph structure. This population graph can then be used for medical downstream tasks using graph neural networks (GNNs). The construction of a suitable graph structure is a challenging step in the learning pipeline that can have severe impact on model performance. To this end, different graph assessment metrics have been introduced to evaluate graph structures. However, these metrics are limited to classification tasks and discrete adjacency matrices, only covering a small subset of real-world applications. In this work, we introduce extended graph assessment metrics (GAMs) for regression tasks and continuous adjacency matrices. We focus on two GAMs in specific: \textit{homophily} and \textit{cross-class neighbourhood similarity} (CCNS). We extend the notion of GAMs to more than one hop, define homophily for regression tasks, as well as continuous adjacency matrices, and propose a light-weight CCNS distance for discrete and continuous adjacency matrices. We show the correlation of these metrics with model performance on different medical population graphs and under different learning settings.