Graffe: Graph Representation Learning via Diffusion Probabilistic Models

TL;DR

Graffe introduces a novel self-supervised diffusion model for graph representation learning, theoretically grounding its effectiveness and achieving state-of-the-art results on multiple graph classification benchmarks.

Contribution

The paper presents Graffe, the first diffusion probabilistic model tailored for graph representation learning, with theoretical analysis and competitive empirical performance.

Findings

The denoising objective maximizes conditional mutual information.

Graffe achieves state-of-the-art results on 9 out of 11 datasets.

Theoretical proof links diffusion models to information maximization.

Abstract

Diffusion probabilistic models (DPMs), widely recognized for their potential to generate high-quality samples, tend to go unnoticed in representation learning. While recent progress has highlighted their potential for capturing visual semantics, adapting DPMs to graph representation learning remains in its infancy. In this paper, we introduce Graffe, a self-supervised diffusion model proposed for graph representation learning. It features a graph encoder that distills a source graph into a compact representation, which, in turn, serves as the condition to guide the denoising process of the diffusion decoder. To evaluate the effectiveness of our model, we first explore the theoretical foundations of applying diffusion models to representation learning, proving that the denoising objective implicitly maximizes the conditional mutual information between data and its representation. Specifically, we prove that the negative logarithm of the denoising score matching loss is a tractable lower bound for the conditional mutual information. Empirically, we conduct a series of case studies to validate our theoretical insights. In addition, Graffe delivers competitive results under the linear probing setting on node and graph classification tasks, achieving state-of-the-art performance on 9 of the 11 real-world datasets. These findings indicate that powerful generative models, especially diffusion models, serve as an effective tool for graph representation learning.