Diffusion-Based Symbolic Regression
Zachary Bastiani, Robert M. Kirby, Jacob Hochhalter, Shandian Zhe
TL;DR
This paper introduces a diffusion-based method for symbolic regression that generates diverse equations using a novel denoising process and reinforcement learning, achieving high-quality results.
Contribution
It presents a new diffusion framework combined with reinforcement learning techniques for symbolic regression, which is a novel approach in this domain.
Findings
Effective generation of diverse equations
Enhanced performance through policy optimization
Validated by extensive experiments and ablation studies
Abstract
Diffusion has emerged as a powerful framework for generative modeling, achieving remarkable success in applications such as image and audio synthesis. Enlightened by this progress, we propose a novel diffusion-based approach for symbolic regression. We construct a random mask-based diffusion and denoising process to generate diverse and high-quality equations. We integrate this generative processes with a token-wise Group Relative Policy Optimization (GRPO) method to conduct efficient reinforcement learning on the given measurement dataset. In addition, we introduce a long short-term risk-seeking policy to expand the pool of top-performing candidates, further enhancing performance. Extensive experiments and ablation studies have demonstrated the effectiveness of our approach.
Peer Reviews
Decision·Submitted to ICLR 2026
1. The paper is generally well-written and clear in its contributions. 2. The topic, finding interpretable analytical relationships, is important and relevant to ICLR. 3. The idea is fairly novel: as far as I know, the combination of Diffusion Models and Symbolic Regression hasn’t been explored before. DDSR seems like a solid and interesting improvement over DSR. 5. There is good empirical validation. Results on the SRBench show that DDSR outperforms the original DSR in symbolic solution rate
1. This isn’t exactly a weakness, but I have doubts whether this model can actually be called a diffusion model. The “diffusion” process here masks one token at a time instead of adding noise to all tokens, and there is no Markov chain. Mathematically it seems different, and it would perhaps be better to call it a diffusion-inspired model. Please correct me if I’ve misunderstood this. 2. The discussion around the 10% noise case is somewhat confusing. The concept of “token space” isn’t previousl
The paper structure is clear and easy to follow. The experiments are extensive.
- Novelty: I fail to see the novelty of this work. The authors claim to apply masked diffusion and GRPO to the symbolic regression task; however, both are well-established frameworks with clear prior formulations. The paper does not highlight any critical adaptations or theoretical insights specific to symbolic regression that would justify a new contribution. Without such task-specific modifications, the method essentially reduces to a straightforward application of existing components. - Long
1. The authors proposed a novel method for symbolic regression that combines diffusion methods and reinforcement learning, providing supportive evaluation against 18 baselines. 2. The method can generate free-form expression with much less tokens comparing to other methods, which is promising. 3. The paper is well-writing, with systematical explanations of experiments via numeric results and figures.
1. Not enough datasets for comprehensive evaluation. In the paper, the authors only use 1 dataset that has 133 problems with ground-truth and 120 problems without solutions, which is relatively small size relative to diffusion models with transformer-based architectures and might incur overfitting problem. The difference among SRbench, Feynman and Strogatz datasets should be stated clearly. 2. The ablation study replaces some key components with other components from prior literature, whereas th
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEvolutionary Algorithms and Applications
MethodsDiffusion