Local Search GFlowNets
Minsu Kim, Taeyoung Yun, Emmanuel Bengio, Dinghuai Zhang, Yoshua, Bengio, Sungsoo Ahn, Jinkyoo Park
TL;DR
This paper enhances Generative Flow Networks by integrating local search techniques to improve the generation of high-reward samples, especially in biochemical applications, addressing over-exploration issues.
Contribution
It introduces a novel training method for GFlowNets using local search via backtracking and reconstruction to bias sampling toward high-reward solutions.
Findings
Significant performance improvements in biochemical tasks.
Effective biasing toward high-reward solutions.
Enhanced sample diversity and quality.
Abstract
Generative Flow Networks (GFlowNets) are amortized sampling methods that learn a distribution over discrete objects proportional to their rewards. GFlowNets exhibit a remarkable ability to generate diverse samples, yet occasionally struggle to consistently produce samples with high rewards due to over-exploration on wide sample space. This paper proposes to train GFlowNets with local search, which focuses on exploiting high-rewarded sample space to resolve this issue. Our main idea is to explore the local neighborhood via backtracking and reconstruction guided by backward and forward policies, respectively. This allows biasing the samples toward high-reward solutions, which is not possible for a typical GFlowNet solution generation scheme, which uses the forward policy to generate the solution from scratch. Extensive experiments demonstrate a remarkable performance improvement in…
Peer Reviews
Decision·ICLR 2024 spotlight
- Novelty: the method is novel as it cleverly takes advantage of both the probabilistic forward and backward policy of GFNs. - Significance: the proposed methods exhibits excellent empirical results and drastically outperforms the other baselines. I wish there were more of them, but more on that below. - Clarity: the paper is well written and provides an exceptionally clear and concise introduction to generative flow networks. Moreover the method is quite simple (I think I could replicate the au
- While the method itself is simple and well presented, it remains a little unclear the relative importance of different components. For example, the exposition focuses on the trajectory balanced objective — does this mean LS-GFNs don’t work with different training objectives? The paper also mentions building on top of Shen et al., 2023 which introduces prioritized replay training (PRT) to GFNs, but this method doesn’t appear as a baseline — so how much of the improvements are due to PRT and how
1. The paper introduces an algorithm which combines inter-mode global exploration with intra-mode local exploration in GFlowNets training. 2. The paper is well-written and easy to understand. The proposed algorithm and experimental setup are clearly described.
1. The paper does not declare the sampling complexity of the proposed method. The local search may require more sampling, which can lead to an unfair comparison. 2. The limitations of the proposed algorithm and potential drawbacks are not discussed in detail.
- The presentation is clear - The idea is simple and easy to implement - The evaluation is comprehensive and showcases the strength of the idea
- The proposed method adds some computational overhead
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCaching and Content Delivery · Cloud Computing and Resource Management · Advanced Image and Video Retrieval Techniques