On Calibration of Graph Neural Networks for Node Classification

TL;DR

This paper investigates the calibration of graph neural networks for node classification, analyzing existing methods and proposing a new topology-aware calibration approach to improve confidence score reliability.

Contribution

It introduces a novel topology-aware calibration method and provides comprehensive analysis of factors affecting GNN calibration, filling a research gap.

Findings

Existing calibration methods have limited effectiveness on GNNs.

Model capacity and graph density influence calibration quality.

The proposed topology-aware method improves calibration accuracy.

Abstract

Graphs can model real-world, complex systems by representing entities and their interactions in terms of nodes and edges. To better exploit the graph structure, graph neural networks have been developed, which learn entity and edge embeddings for tasks such as node classification and link prediction. These models achieve good performance with respect to accuracy, but the confidence scores associated with the predictions might not be calibrated. That means that the scores might not reflect the ground-truth probabilities of the predicted events, which would be especially important for safety-critical applications. Even though graph neural networks are used for a wide range of tasks, the calibration thereof has not been sufficiently explored yet. We investigate the calibration of graph neural networks for node classification, study the effect of existing post-processing calibration methods, and analyze the influence of model capacity, graph density, and a new loss function on calibration. Further, we propose a topology-aware calibration method that takes the neighboring nodes into account and yields improved calibration compared to baseline methods.