SymDiff: Equivariant Diffusion via Stochastic Symmetrisation
Leo Zhang, Kianoosh Ashouritaklimi, Yee Whye Teh, Rob Cornish
TL;DR
SymDiff introduces a lightweight, efficient method for creating equivariant diffusion models through stochastic symmetrisation, enhancing molecular generation without complex neural network modifications.
Contribution
It presents the first application of symmetrisation in generative modeling, enabling scalable, flexible equivariant diffusion models without specialized neural network components.
Findings
Improved molecular generation performance with SymDiff.
No need for complex equivariant neural network architectures.
Applicable as a drop-in enhancement for existing models.
Abstract
We propose SymDiff, a method for constructing equivariant diffusion models using the framework of stochastic symmetrisation. SymDiff resembles a learned data augmentation that is deployed at sampling time, and is lightweight, computationally efficient, and easy to implement on top of arbitrary off-the-shelf models. In contrast to previous work, SymDiff typically does not require any neural network components that are intrinsically equivariant, avoiding the need for complex parameterisations or the use of higher-order geometric features. Instead, our method can leverage highly scalable modern architectures as drop-in replacements for these more constrained alternatives. We show that this additional flexibility yields significant empirical benefit for -equivariant molecular generation. To the best of our knowledge, this is the first application of symmetrisation to…
Peer Reviews
Decision·ICLR 2025 Poster
**Clarity**: The paper is overall well written and the necessary background is introduced in an appropriate manner, such that one can follow the overall paper. However, the stochastic symmetrization framework is introduced in a fairly abstract manner and I believe more intuitions could be discussed and examples given, which would likely help the broad ICLR audience (see questions below). **Originality**: Building equivariant diffusion models with stochastic symmetrization is an overall novel an
I also see some weaknesses. - *More discussion of intuitions and more experimental analyses*: I think it would be great if the authors discussed some intuitions in more detail and analyzed their models in more detail. Specifically: - In practice, training SymDiff boils down to the training procedure in Algorithm 1. $\gamma_\theta$ is trained as part of the ELBO objective, using reparametrization. What sort of $\gamma_\theta$ and $f_\theta$ functions are we intuitively expected to learn in dif
This paper introduces SYMDIFF, a simple yet powerful method for constructing equivariant diffusion models without relying on complex intrinsically equivariant neural network architectures. The strengths of the paper can be assessed across the following dimensions: Originality: The paper presents a novel application of stochastic symmetrisation to generative modeling, specifically in the context of diffusion models. This approach is original because it departs from traditional methods that requi
While the paper introduces SYMDIFF as a flexible and efficient method for constructing equivariant diffusion models, it lacks specific guidelines to help practitioners decide when to use SYMDIFF over traditional intrinsically equivariant models. This absence of decision-making support makes it challenging to understand the circumstances under which SYMDIFF is more advantageous. To address this, I suggest the authors include a comparative analysis between SYMDIFF and traditional methods. This co
- Originality: The paper proposes SymDiff, a novel method using stochastic symmetrisation for equivariant diffusion models, extending symmetrisation to generative modelling and differing from prior approaches. - Clarity: The paper is well-structured, clearly explaining concepts with examples and presenting algorithms, aiding understanding and implementation. - Quality: The paper presents a comprehensive framework with solid derivations and conducts rigorous experiments on multiple datasets.
- Performance gap: from Tab2 and 3, seems the model is not parameter efficient compared with baseline models. When reducing the model size, the performance will have a larger drop than the baseline model. - Unfair comparison: I didn't fully understand for SymDiff+++, ++, and +, why you cannot choose the same number of parameters of EDM. It seems your model SymDiff+ is already larger than EDM+++, but with worse performance. - Missing comparison: I think the author also needs to incorporate GeoL
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Mathematical Biology Tumor Growth · Model Reduction and Neural Networks
MethodsDiffusion