Boosted GFlowNets: Improving Exploration via Sequential Learning

TL;DR

Boosted GFlowNets enhance exploration in generative sampling by sequentially training ensembles to focus on underexplored regions, leading to better coverage of high-reward areas and improved sample diversity.

Contribution

The paper introduces Boosted GFlowNets, a novel ensemble-based method that sequentially trains models to address exploration imbalance in GFlowNets.

Findings

Achieves better exploration and diversity on synthetic benchmarks.

Maintains stability and simplicity of standard training methods.

Ensures non-degradation of learned distributions.

Abstract

Generative Flow Networks (GFlowNets) are powerful samplers for compositional objects that, by design, sample proportionally to a given non-negative reward. Nonetheless, in practice, they often struggle to explore the reward landscape evenly: trajectories toward easy-to-reach regions dominate training, while hard-to-reach modes receive vanishing or uninformative gradients, leading to poor coverage of high-reward areas. We address this imbalance with Boosted GFlowNets, a method that sequentially trains an ensemble of GFlowNets, each optimizing a residual reward that compensates for the mass already captured by previous models. This residual principle reactivates learning signals in underexplored regions and, under mild assumptions, ensures a monotone non-degradation property: adding boosters cannot worsen the learned distribution and typically improves it. Empirically, Boosted GFlowNets achieve substantially better exploration and sample diversity on multimodal synthetic benchmarks and peptide design tasks, while preserving the stability and simplicity of standard trajectory-balance training.