Bias mitigation in graph diffusion models

TL;DR

This paper identifies bias issues in graph diffusion models and introduces a comprehensive, network-free mitigation approach using Langevin sampling and score correction, validated across various models and datasets.

Contribution

It proposes a novel bias mitigation method for graph diffusion models that aligns reverse sampling with forward perturbations and corrects score differences without network modifications.

Findings

Achieves state-of-the-art results on multiple datasets.

Effectively mitigates reverse-starting and exposure biases.

Validated across various models and tasks.

Abstract

Most existing graph diffusion models have significant bias problems. We observe that the forward diffusion's maximum perturbation distribution in most models deviates from the standard Gaussian distribution, while reverse sampling consistently starts from a standard Gaussian distribution, which results in a reverse-starting bias. Together with the inherent exposure bias of diffusion models, this results in degraded generation quality. This paper proposes a comprehensive approach to mitigate both biases. To mitigate reverse-starting bias, we employ a newly designed Langevin sampling algorithm to align with the forward maximum perturbation distribution, establishing a new reverse-starting point. To address the exposure bias, we introduce a score correction mechanism based on a newly defined score difference. Our approach, which requires no network modifications, is validated across multiple models, datasets, and tasks, achieving state-of-the-art results.Code is at https://github.com/kunzhan/spp