Exploring Multiple High-Scoring Subspaces in Generative Flow Networks

TL;DR

This paper introduces CMAB-GFN, a method combining combinatorial multi-armed bandits with GFlowNets to focus exploration on high-reward subspaces, improving the quality of generated solutions in complex sampling tasks.

Contribution

The paper proposes a novel integration of CMAB with GFlowNets to efficiently identify and explore high-scoring subspaces, enhancing sampling quality and diversity.

Findings

CMAB-GFN outperforms existing methods in generating higher-reward candidates.

The approach accelerates discovery of high-value solutions.

It maintains diversity while focusing on promising regions.

Abstract

As a probabilistic sampling framework, Generative Flow Networks (GFlowNets) show strong potential for constructing complex combinatorial objects through the sequential composition of elementary components. However, existing GFlowNets often suffer from excessive exploration over vast state spaces, leading to over-sampling of low-reward regions and convergence to suboptimal distributions. Effectively biasing GFlowNets toward high-reward solutions remains a non-trivial challenge. In this paper, we propose CMAB-GFN, which integrates a combinatorial multi-armed bandit (CMAB) framework with GFlowNet policies. The CMAB component prunes low-quality actions, yielding compact high-scoring subspaces for exploration. Restricting GFNs to these compact high-scoring subspaces accelerates the discovery of high-value candidates, while the exploration of different subspaces ensures that diversity is not sacrificed. Experimental results on multiple tasks demonstrate that CMAB-GFN generates higher-reward candidates than existing approaches.