Graph Representation Learning with Diffusion Generative Models

TL;DR

This paper explores the application of discrete diffusion models within an autoencoder framework to learn meaningful embeddings for graph-structured data, addressing the challenge of adapting diffusion processes to discrete graphs.

Contribution

It introduces a novel method combining discrete diffusion models with autoencoders for effective graph representation learning, a relatively unexplored area.

Findings

Successful autoencoding of graph data using diffusion models

Effective extraction of graph embeddings from diffusion process

Potential for improved graph representation learning

Abstract

Diffusion models have established themselves as state-of-the-art generative models across various data modalities, including images and videos, due to their ability to accurately approximate complex data distributions. Unlike traditional generative approaches such as VAEs and GANs, diffusion models employ a progressive denoising process that transforms noise into meaningful data over multiple iterative steps. This gradual approach enhances their expressiveness and generation quality. Not only that, diffusion models have also been shown to extract meaningful representations from data while learning to generate samples. Despite their success, the application of diffusion models to graph-structured data remains relatively unexplored, primarily due to the discrete nature of graphs, which necessitates discrete diffusion processes distinct from the continuous methods used in other domains. In this work, we leverage the representational capabilities of diffusion models to learn meaningful embeddings for graph data. By training a discrete diffusion model within an autoencoder framework, we enable both effective autoencoding and representation learning tailored to the unique characteristics of graph-structured data. We extract the representation from the combination of the encoder's output and the decoder's first time step hidden embedding. Our approach demonstrates the potential of discrete diffusion models to be used for graph representation learning. The code can be found at https://github.com/DanielMitiku/Graph-Representation-Learning-with-Diffusion-Generative-Models