Equivariant Denoisers Cannot Copy Graphs: Align Your Graph Diffusion Models
Najwa Laabid, Severi Rissanen, Markus Heinonen, Arno Solin, Vikas Garg
TL;DR
This paper reveals the limitations of standard equivariant denoisers in graph diffusion models for graph-to-graph translation and proposes an alignment method that significantly improves performance in chemical reaction prediction.
Contribution
The paper introduces an alignment technique to overcome symmetry limitations in permutation equivariant denoisers for graph diffusion models, boosting accuracy in retrosynthesis tasks.
Findings
Alignment improves top-1 accuracy from 5% to 54.7%.
Permutation equivariant denoisers cannot effectively copy graphs due to symmetry constraints.
Proposed method achieves state-of-the-art performance in chemical reaction prediction.
Abstract
Graph diffusion models, dominant in graph generative modeling, remain underexplored for graph-to-graph translation tasks like chemical reaction prediction. We demonstrate that standard permutation equivariant denoisers face fundamental limitations in these tasks due to their inability to break symmetries in noisy inputs. To address this, we propose aligning input and target graphs to break input symmetries while preserving permutation equivariance in non-matching graph portions. Using retrosynthesis (i.e., the task of predicting precursors for synthesis of a given target molecule) as our application domain, we show how alignment dramatically improves discrete diffusion model performance from 5% to a SOTA-matching 54.7% top-1 accuracy. Code is available at https://github.com/Aalto-QuML/DiffAlign.
Peer Reviews
Decision·ICLR 2025 Poster
- The paper addresses a novel and relevant problem in graph diffusion literature, particularly in breaking symmetries for graph-to-graph translation, which has not been explored in depth. - The idea of using equivariant models and tackling their limitations is well-motivated, and the theoretical underpinning is clear and compelling. - The alignment techniques proposed for mapping nodes between input and output graphs are intuitive and demonstrate promising preliminary results, especially compare
- The justification for certain design choices, such as using the absorbing state distribution instead of marginal distributions (as in DiGress), is insufficiently explained. A more thorough motivation or comparison would be beneficial. - Some key claims, particularly in Section 3.2, are vague or lack detailed explanation. For example, the statement regarding the model's capability to handle graph transformations requires clearer elaboration. - The experimental validation, while promising, lacks
The paper presents a comprehensive set of experiments that demonstrate the benefits and the practical utility of the proposed model in the context of guided and conditional generation of molecular graphs. The model achieves state-of-the-art results in chemical retrosynthesis. The paper concerns an interesting problem, and its central message is important to the community. Nice and clear illustrations accompany the ideas introduced in the paper.
The writing deserves improvement. Certain parts of the paper need to be communicated more clearly. The notation sometimes needs to be clarified. These concerns are detailed in the following comments. Major: The content from the start of Section 3 up to Section 3.2 qualifies more like a background section. The material is heavily inspired by [1]. Please note that [1] also uses conditioning, though it is not presented in the equations in such an explicit way. It takes more work to recognize the
- To the best of my knowledge, the topic of breaking symmetries to perform graph-to-graph translation has not yet been addressed in the graph diffusion literature. - The problem of using equivariant models is clearly identified and backed by theoritical evidences. - The paper proposed several simple ways to solve this problem.
- Why using the aborbing state distribution and not the marginal distributions introduced in DiGress (Vignac et al., 2022) ? - I would avoid statements like "it is easy to see" l304 or "Clearly" l242 and provide instead clear motivations and explanations - Some claims or design choices lack motivation or justification : - For example, the claims in section 3.2 "a model that is capable of copying graphs from one side of the reaction to the other should also be extendable to modulations of this
Code & Models
Videos
Taxonomy
TopicsAdvanced Graph Neural Networks · Graph Theory and Algorithms · Bioinformatics and Genomic Networks
MethodsSPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings · Diffusion