Conditional Prediction ROC Bands for Graph Classification

TL;DR

This paper introduces CP-ROC bands, a method for uncertainty quantification in ROC curves for graph classification, ensuring reliability under distributional shifts, especially in non-iid test data scenarios.

Contribution

The paper develops CP-ROC bands with guaranteed coverage for ROC curves, adaptable to various GNNs, and introduces a data-driven approach for local exchangeability calibration.

Findings

CP-ROC improves uncertainty quantification for ROC curves.

The method is robust to distributional shifts in test data.

Empirical results show enhanced prediction reliability across tasks.

Abstract

Graph classification in medical imaging and drug discovery requires accuracy and robust uncertainty quantification. To address this need, we introduce Conditional Prediction ROC (CP-ROC) bands, offering uncertainty quantification for ROC curves and robustness to distributional shifts in test data. Although developed for Tensorized Graph Neural Networks (TGNNs), CP-ROC is adaptable to general Graph Neural Networks (GNNs) and other machine learning models. We establish statistically guaranteed coverage for CP-ROC under a local exchangeability condition. This addresses uncertainty challenges for ROC curves under non-iid setting, ensuring reliability when test graph distributions differ from training data. Empirically, to establish local exchangeability for TGNNs, we introduce a data-driven approach to construct local calibration sets for graphs. Comprehensive evaluations show that CP-ROC significantly improves prediction reliability across diverse tasks. This method enhances uncertainty quantification efficiency and reliability for ROC curves, proving valuable for real-world applications with non-iid objects.