Generative Flow Networks: a Markov Chain Perspective

TL;DR

This paper presents a new perspective on Generative Flow Networks (GFlowNets) by formalizing their connection to Markov chains, unifying their theoretical framework with Markov chain Monte Carlo methods and highlighting their similarities.

Contribution

It introduces a Markov chain perspective for GFlowNets, providing a unified theoretical framework that connects them with MCMC methods.

Findings

GFlowNets can be viewed as recurrent Markov chains.

The paper unifies GFlowNets and MCMC under a common theoretical framework.

Insights into the similarities and differences between GFlowNets and MCMC methods.

Abstract

While Markov chain Monte Carlo methods (MCMC) provide a general framework to sample from a probability distribution defined up to normalization, they often suffer from slow convergence to the target distribution when the latter is highly multi-modal. Recently, Generative Flow Networks (GFlowNets) have been proposed as an alternative framework to mitigate this issue when samples have a clear compositional structure, by treating sampling as a sequential decision making problem. Although they were initially introduced from the perspective of flow networks, the recent advances of GFlowNets draw more and more inspiration from the Markov chain literature, bypassing completely the need for flows. In this paper, we formalize this connection and offer a new perspective for GFlowNets using Markov chains, showing a unifying view for GFlowNets regardless of the nature of the state space as recurrent Markov chains. Positioning GFlowNets under the same theoretical framework as MCMC methods also allows us to identify the similarities between both frameworks, and most importantly to highlight their