TL;DR
SESaMo introduces a novel stochastic modulation technique to incorporate symmetries into normalizing flows, improving their flexibility and effectiveness in modeling complex distributions in physics and chemistry.
Contribution
The paper proposes SESaMo, a new method for embedding symmetries into normalizing flows via stochastic modulation, enhancing their ability to learn both exact and broken symmetries.
Findings
Effective in modeling 8-Gaussian mixture distributions
Successfully applied to $ 4$4 field theory and Hubbard model
Benchmark results show improved symmetry learning
Abstract
Deep generative models have recently garnered significant attention across various fields, from physics to chemistry, where sampling from unnormalized Boltzmann-like distributions represents a fundamental challenge. In particular, autoregressive models and normalizing flows have become prominent due to their appealing ability to yield closed-form probability densities. Moreover, it is well-established that incorporating prior knowledge - such as symmetries - into deep neural networks can substantially improve training performances. In this context, recent advances have focused on developing symmetry-equivariant generative models, achieving remarkable results. Building upon these foundations, this paper introduces Symmetry-Enforcing Stochastic Modulation (SESaMo). Similar to equivariant normalizing flows, SESaMo enables the incorporation of inductive biases (e.g., symmetries) into…
Peer Reviews
Decision·ICLR 2026 Poster
- The paper introduces a systematic way to incorporate symmetries in NF. - Experimental testbed is good. The complex $\phi^4$ and Hubbard models are known to be challenging distributions. - The idea of using a penalty term to enforce bijectivity is interesting. - The paper is well written and is mostly comprehensible.
I think several essential questions/issues are overlooked both in theory and experiments. - Experimental results report only ESS and RKL (reverse KL divergence). However, the performance on two metrics can be good (i.e., high ESS and low RKL) even in the case of mode collapse (see [AdvNF 2025]), i.e., when the model learns and explores only a part of the target distribution. A standard metric robust to this problem is the negative log-likelihood (NLL). - The baselines are weak. The state of the
* The proposed method is a simple and flexible way to impose both exact and broken symmetries on NFs without requiring explicitly equivariant architectures or canonicalization. * The work is technically sound and supported with solid empirical validation. * Furthermore, the proposed method integrates naturally with standard NFs (e.g. RealNVP) and requires relatively minimal overhead (a REINFORCE estimator to enable gradients through a stochastic variable).
* While the proposed method is elegant in principle, the exposition is overly abstract which makes the main idea underlying the method difficult to understand. For example, the authors provide several intuitive examples in increasing generality, but relegate them to the appendix (E, F). The paper could benefit from a clear algorithm box/pseudocode implementation describing the key components (flow, modulation map, and probability weights) and, in increasing generality, how each can be set or aug
The paper proposes a creative mechanism to integrate group symmetries into flow-based generative models using stochastic modulation. This differs from previous deterministic equivariant designs (e.g., Equivariant Flows) by introducing a learnable mixture structure that explicitly models symmetry breaking. Experiments demonstrate that SESaMo achieves competitive or superior ESS on discrete-symmetry datasets while maintaining interpretability of the learned symmetry-breaking parameter b.
1. The bijectivity penalty used to enforce separability is heuristic and not theoretically justified. The experimental comparisons only include RealNVP, VMoNF, and canonicalization methods. These are several years old and do not represent the current state of equivariant or symmetry-aware generative modeling. Modern approaches, including equivariant flow mathcing and equivariant diffusion models, should at least be discussed or justified as not directly applicable. Without this clarification, it
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsSoftmax · Attention Is All You Need · Normalizing Flows
