TL;DR
Cometh is a novel continuous-time discrete-state graph diffusion model that improves graph generation quality and efficiency by integrating continuous time and a simplified encoding scheme, outperforming existing models on multiple benchmarks.
Contribution
It introduces a continuous-time discrete-state graph diffusion model with a single random-walk encoding, enhancing expressiveness and performance in graph generation tasks.
Findings
Significant performance improvements over state-of-the-art models.
Achieves 99.5% accuracy on planar graph dataset.
Outperforms DiGress by 12.6% on GuacaMol dataset.
Abstract
Discrete-state denoising diffusion models led to state-of-the-art performance in graph generation, especially in the molecular domain. Recently, they have been transposed to continuous time, allowing more flexibility in the reverse process and a better trade-off between sampling efficiency and quality. Here, to leverage the benefits of both approaches, we propose Cometh, a continuous-time discrete-state graph diffusion model, tailored to the specificities of graph data. In addition, we also successfully replaced the set of structural encodings previously used in the discrete graph diffusion model with a single random-walk-based encoding, providing a simple and principled way to boost the model's expressive power. Empirically, we show that integrating continuous time leads to significant improvements across various metrics over state-of-the-art discrete-state diffusion models on a large…
Peer Reviews
Decision·Submitted to ICLR 2025
1. The discrete-state continuous-time framework for graph generation offers flexibility and is well-suited to the structural properties of graphs. 2. The authors conducted extensive experiments across datasets of varying scales, performed ablation studies, and presented results for conditional generation.
1. In lines 70-71, the authors claim that this is the first work using a continuous-time discrete-state diffusion model for graph generation. However, this is inaccurate as there is a concurrent work, [R1] (Xu et al., 2024). While a direct comparison is not required for Arxiv paper, it would enhance the paper's completeness to mention this work in the submission. [R1] Xu, Zhe, et al. "Discrete-state Continuous-time Diffusion for Graph Generation." arXiv preprint arXiv:2405.11416 (2024). 2. The
1. This paper have achieved state-of-the-art results on synthetic and one of the molecular benchmarks. 2. It makes notable machine learning contributions by adapting continuous-time discrete diffusion from [1] to develop a rate matrix, resulting in a noise model of marginal transition. The paper proposes using RRWP features to effectively model the graph, and appears to enhance performance to some extent.
1. From a machine learning perspective, the contribution of the proposed extension for marginal transition seems minor, and its impact on improving graph generation remains unclear. 2. While the RRWP feature seems to enhance generation performance, the lack of an ablation study makes it difficult to determine the extent to which each component contributes and which factors are responsible for the improvements. 3. There are inconsistencies in the abbreviations "RWPP" and "RRWP" on lines #314 and
- **Clarity of presentation**: the paper has a clear goal (introducing continuous time discrete-state graph diffusion to enable additional sampling methods) and suggests a suitable approach to achieve it (adapting the continuous-time framework proposed by Campbell et al. (2022) [2]) - **Empirical improvement**: COMETH seems to be competitive with existing baselines, achieving better results on at least a couple of metrics in all experiments.
The main weakness of this work is its incremental nature. Below I analyze the contributions of the work and explain my assessment. **TL;DR** Since the continuous time framework for discrete data has already been proposed, a good direction for the present work is to provide a thorough evaluation of the design choices relevant to adapting this framework to graph diffusion. This can include for example both theoretical and empirical evaluations of the choice of loss function, noise schedule, and s
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsSparse Evolutionary Training · Diffusion
