Same Graph, Different Likelihoods: Calibration of Autoregressive Graph Generators via Permutation-Equivalent Encodings

TL;DR

This paper investigates the calibration issues of autoregressive graph generators, introducing a measure called Linearization Uncertainty (LU) to evaluate the consistency of likelihoods across different graph linearizations, and demonstrates its effectiveness in molecular graph generation.

Contribution

The authors propose Linearization Uncertainty as a new metric for assessing likelihood calibration in graph generators and show its advantages over traditional NLL-based evaluation.

Findings

Biased orderings have lower NLL on native linearizations but higher calibration error under permutations.

LU correlates strongly with molecular stability, outperforming NLL in quality assessment.

Permutation-based evaluation provides a more reliable measure of generated graph quality.

Abstract

Autoregressive graph generators define likelihoods via a sequential construction process, but these likelihoods are only meaningful if they are consistent across all linearizations of the same graph. Segmented Eulerian Neighborhood Trails (SENT), a recent linearization method, converts graphs into sequences that can be perfectly decoded and efficiently processed by language models, but admit multiple equivalent linearizations of the same graph. We quantify violations in assigned negative log-likelihood (NLL) using the coefficient of variation across equivalent linearizations, which we call Linearization Uncertainty (LU). Training transformers under four linearization strategies on two datasets, we show that biased orderings achieve lower NLL on their native order but exhibit expected calibration error (ECE) two orders of magnitude higher under random permutation, indicating that these models have learned their training linearization rather than the underlying graph. On the molecular graph benchmark QM9, NLL for generated graphs is negatively correlated with molecular stability (AUC $=0.43$), while LU achieves AUC $=0.85$, suggesting that permutation-based evaluation provides a more reliable quality check for generated molecules. Code is available at https://github.com/lauritsf/linearization-uncertainty