On Generalization for Generative Flow Networks

TL;DR

This paper formalizes the concept of generalization in Generative Flow Networks (GFlowNets), linking it to stability and evaluating their ability to generalize to unseen, longer trajectories in the reward function.

Contribution

It provides a formal framework for understanding generalization in GFlowNets and designs experiments to assess their capacity to generalize to longer, unseen trajectories.

Findings

GFlowNets can generalize to longer trajectories than seen in training

Formal link between generalization and stability in GFlowNets

Experimental evidence on length generalization capabilities

Abstract

Generative Flow Networks (GFlowNets) have emerged as an innovative learning paradigm designed to address the challenge of sampling from an unnormalized probability distribution, called the reward function. This framework learns a policy on a constructed graph, which enables sampling from an approximation of the target probability distribution through successive steps of sampling from the learned policy. To achieve this, GFlowNets can be trained with various objectives, each of which can lead to the model s ultimate goal. The aspirational strength of GFlowNets lies in their potential to discern intricate patterns within the reward function and their capacity to generalize effectively to novel, unseen parts of the reward function. This paper attempts to formalize generalization in the context of GFlowNets, to link generalization with stability, and also to design experiments that assess the capacity of these models to uncover unseen parts of the reward function. The experiments will focus on length generalization meaning generalization to states that can be constructed only by longer trajectories than those seen in training.