On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction

TL;DR

This paper investigates how introducing a temperature parameter into Bayesian GNNs within the conformal prediction framework can improve the efficiency of uncertainty quantification, providing more precise prediction sets.

Contribution

It demonstrates empirically that optimal temperature tuning enhances the efficiency of conformal prediction sets for Bayesian GNNs, and analyzes factors affecting their calibration and performance.

Findings

Existence of temperatures that improve prediction set efficiency

Insights into calibration factors affecting conformal prediction

Analysis of model and data influences on prediction set size

Abstract

Accurate uncertainty quantification in graph neural networks (GNNs) is essential, especially in high-stakes domains where GNNs are frequently employed. Conformal prediction (CP) offers a promising framework for quantifying uncertainty by providing $\textit{valid}$ prediction sets for any black-box model. CP ensures formal probabilistic guarantees that a prediction set contains a true label with a desired probability. However, the size of prediction sets, known as $\textit{inefficiency}$, is influenced by the underlying model and data generating process. On the other hand, Bayesian learning also provides a credible region based on the estimated posterior distribution, but this region is $\textit{well-calibrated}$ only when the model is correctly specified. Building on a recent work that introduced a scaling parameter for constructing valid credible regions from posterior estimate, our study explores the advantages of incorporating a temperature parameter into Bayesian GNNs within CP framework. We empirically demonstrate the existence of temperatures that result in more efficient prediction sets. Furthermore, we conduct an analysis to identify the factors contributing to inefficiency and offer valuable insights into the relationship between CP performance and model calibration.