Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy

TL;DR

This paper introduces Simi-Mailbox, a novel GNN calibration method that categorizes nodes by neighborhood similarity and confidence, enabling fine-grained, group-specific temperature scaling to improve prediction calibration accuracy.

Contribution

The paper proposes Simi-Mailbox, a new approach that refines GNN calibration by grouping nodes based on both neighborhood similarity and confidence, moving beyond the assumption of uniform calibration within neighborhood groups.

Findings

Achieves up to 13.79% error reduction in calibration errors.

Demonstrates effectiveness across diverse datasets and GNN architectures.

Reveals limitations of existing neighborhood similarity-based calibration methods.

Abstract

Recent advancements in graph neural networks (GNNs) have highlighted the critical need of calibrating model predictions, with neighborhood prediction similarity recognized as a pivotal component. Existing studies suggest that nodes with analogous neighborhood prediction similarity often exhibit similar calibration characteristics. Building on this insight, recent approaches incorporate neighborhood similarity into node-wise temperature scaling techniques. However, our analysis reveals that this assumption does not hold universally. Calibration errors can differ significantly even among nodes with comparable neighborhood similarity, depending on their confidence levels. This necessitates a re-evaluation of existing GNN calibration methods, as a single, unified approach may lead to sub-optimal calibration. In response, we introduce **Simi-Mailbox**, a novel approach that categorizes nodes by both neighborhood similarity and their own confidence, irrespective of proximity or connectivity. Our method allows fine-grained calibration by employing *group-specific* temperature scaling, with each temperature tailored to address the specific miscalibration level of affiliated nodes, rather than adhering to a uniform trend based on neighborhood similarity. Extensive experiments demonstrate the effectiveness of our **Simi-Mailbox** across diverse datasets on different GNN architectures, achieving up to 13.79\% error reduction compared to uncalibrated GNN predictions.