Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions?

TL;DR

This paper investigates the ability of graph diffusion models to accurately generate substructure distributions, revealing limitations of current models and proposing more expressive GNN backbones for improved performance.

Contribution

It identifies the expressivity limitations of existing graph diffusion models and introduces a theoretical framework linking GNN expressivity to distribution capturing ability, leading to enhanced models.

Findings

Existing models fail to replicate substructure distributions.

More expressive GNN backbones improve substructure generation.

Theoretical link between GNN expressivity and diffusion model performance.

Abstract

Diffusion models have gained popularity in graph generation tasks; however, the extent of their expressivity concerning the graph distributions they can learn is not fully understood. Unlike models in other domains, popular backbones for graph diffusion models, such as Graph Transformers, do not possess universal expressivity to accurately model the distribution scores of complex graph data. Our work addresses this limitation by focusing on the frequency of specific substructures as a key characteristic of target graph distributions. When evaluating existing models using this metric, we find that they fail to maintain the distribution of substructure counts observed in the training set when generating new graphs. To address this issue, we establish a theoretical connection between the expressivity of Graph Neural Networks (GNNs) and the overall performance of graph diffusion models, demonstrating that more expressive GNN backbones can better capture complex distribution patterns. By integrating advanced GNNs into the backbone architecture, we achieve significant improvements in substructure generation.