Partial GFlowNet: Accelerating Convergence in Large State Spaces via Strategic Partitioning

TL;DR

This paper introduces Partial GFlowNet, a method that accelerates convergence in large state spaces by partitioning the space and strategically exploring high-reward regions, improving both efficiency and diversity.

Contribution

It proposes a novel partitioning and heuristic exploration strategy for GFlowNets to enhance convergence speed and candidate diversity in large state spaces.

Findings

Faster convergence than existing methods on large state spaces

Generates higher reward candidates with increased diversity

Effective partitioning improves exploration efficiency

Abstract

Generative Flow Networks (GFlowNets) have shown promising potential to generate high-scoring candidates with probability proportional to their rewards. As existing GFlowNets freely explore in state space, they encounter significant convergence challenges when scaling to large state spaces. Addressing this issue, this paper proposes to restrict the exploration of actor. A planner is introduced to partition the entire state space into overlapping partial state spaces. Given their limited size, these partial state spaces allow the actor to efficiently identify subregions with higher rewards. A heuristic strategy is introduced to switch partial regions thus preventing the actor from wasting time exploring fully explored or low-reward partial regions. By iteratively exploring these partial state spaces, the actor learns to converge towards the high-reward subregions within the entire state space. Experiments on several widely used datasets demonstrate that \modelname converges faster than existing works on large state spaces. Furthermore, \modelname not only generates candidates with higher rewards but also significantly improves their diversity.