Unlocking Guidance for Discrete State-Space Diffusion and Flow Models
Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, Jennifer, Listgarten
TL;DR
This paper introduces Discrete Guidance, a novel method for applying guidance techniques to discrete state-space diffusion and flow models, enabling controlled generation in natural sciences applications.
Contribution
The paper presents a general, principled approach leveraging continuous-time Markov processes for guidance in discrete models, addressing a key gap in existing methods.
Findings
Effective guided generation of small-molecules
Successful application to DNA sequence design
Demonstrated utility in protein sequence generation
Abstract
Generative models on discrete state-spaces have a wide range of potential applications, particularly in the domain of natural sciences. In continuous state-spaces, controllable and flexible generation of samples with desired properties has been realized using guidance on diffusion and flow models. However, these guidance approaches are not readily amenable to discrete state-space models. Consequently, we introduce a general and principled method for applying guidance on such models. Our method depends on leveraging continuous-time Markov processes on discrete state-spaces, which unlocks computational tractability for sampling from a desired guided distribution. We demonstrate the utility of our approach, Discrete Guidance, on a range of applications including guided generation of small-molecules, DNA sequences and protein sequences.
Peer Reviews
Decision·ICLR 2025 Poster
The authors of this paper propose a guidance framework for generating models (diffusion models, flow matching models) on discrete state Spaces. This guidance strategy is applicable to a wide range of generation models of discrete state Spaces achieved through CTMC, and has been applied to the generation of guidance conditions in many fields such as small molecules, DNA sequences, and protein sequences. In addition, this paper has a reasonable logical structure and clear expression.
While the authors have demonstrated that this guiding framework is empirically effective, more precise theoretical guarantees, such as error analysis, are lacking, and further research into potential tradeoffs between efficiency and accuracy will be of interest to the community. In addition, the article is more inclined to show the effect of guidance and ignore the quality of generation, so it is difficult for readers to judge the practicality of this guidance framework. Finally, judging from th
- The paper proposes a general method for applying guidance on discrete state-space diffusion models, which constitutes a novel contribution to the field. The authors provide a comprehensive derivation of the predictor-guided and predictor-free-guided rate matrices for the discrete diffusion model, accompanied by intuitive explanations of how they compare with the continuous guidance mechanism. - The proposed method is demonstrated through its successful application to a diverse range of applica
- The paper omits an introduction to the forward and backward processes of the discrete state-space diffusion models. Additionally, the rationale behind the training of the rate matrix $R_t(x,\tilde x)$ is not explicitly provided. I recommend that the authors commence with a more general introduction to the discrete state-space diffusion models, followed by a detailed explanation of the guidance mechanism in the continuous case, with a clear comparison between the two. - Furthermore, it would be
The paper introduces a principle mechanism for guidance in a discrete state flow-based generative model. Experiments demonstrate the feasibility of the approach in three different settings (and small images Appendix F.1). Reproducibility is ensured by providing source code. Overall I find that the paper is too dense for the ICLR format and would benefit from a more in depth review. Yet the approach is certainly of interest for the community.
The term coined "discrete guidance" seems too general since there is already another competing approach called DiGress (Vignac et al. ICLR 2023) and also Dirichlet FM, sometimes with similar performance (eg 423 "As predictor-free guidance for DirFM-CFG and DG-PFG behaved comparably (Appendix F.3.5)") In that respect, the empirical results being relatively close to DiGress, the title of the paper is also a bit problematic in my opinion. The mathematical presentation in the main paper is a too q
Videos
Taxonomy
TopicsSimulation Techniques and Applications
MethodsDiffusion