Improving Generative Flow Networks with Path Regularization

TL;DR

This paper introduces a path regularization technique based on optimal transport theory to enhance exploration and generalization in Generative Flow Networks, leading to more diverse and better-structured generated objects.

Contribution

The work proposes a novel path regularization method for GFlowNets using optimal transport, with efficient implementation and theoretical bounds, improving their exploration and generalization capabilities.

Findings

Enhanced diversity and novelty in generated candidates

Improved exploration in active learning scenarios

Better modeling of complex distributions

Abstract

Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability proportional to a given reward function. The central problem of GFlowNets is to improve their exploration and generalization. In this work, we propose a novel path regularization method based on optimal transport theory that places prior constraints on the underlying structure of the GFlowNets. The prior is designed to help the GFlowNets better discover the latent structure of the target distribution or enhance its ability to explore the environment in the context of active learning. The path regularization controls the flow in GFlowNets to generate more diverse and novel candidates via maximizing the optimal transport distances between two forward policies or to improve the generalization via minimizing the optimal transport distances. In addition, we derive an efficient implementation of the regularization by finding its closed form solutions in specific cases and a meaningful upper bound that can be used as an approximation to minimize the regularization term. We empirically demonstrate the advantage of our path regularization on a wide range of tasks, including synthetic hypergrid environment modeling, discrete probabilistic modeling, and biological sequence design.