Training-free Composition of Pre-trained GFlowNets for Multi-Objective Generation

TL;DR

This paper introduces a training-free method to combine pre-trained GFlowNets for multi-objective generation, enabling quick adaptation to new reward combinations without retraining, suitable for scientific discovery tasks.

Contribution

It proposes a novel inference-time mixing policy for GFlowNets that handles diverse multi-objective reward functions without additional training.

Findings

Achieves comparable performance to trained baselines in experiments

Exact recovery for linear scalarization of objectives

Quantifies approximation quality for nonlinear reward operators

Abstract

Generative Flow Networks (GFlowNets) learn to sample diverse candidates in proportion to a reward function, making them well-suited for scientific discovery, where exploring multiple promising solutions is crucial. Further extending GFlowNets to multi-objective settings has attracted growing interest since real-world applications often involve multiple, conflicting objectives. However, existing approaches require additional training for each set of objectives, limiting their applicability and incurring substantial computational overhead. We propose a training-free mixing policy that composes pre-trained GFlowNets at inference time, enabling rapid adaptation without finetuning or retraining. Importantly, our framework is flexible, capable of handling diverse reward combinations ranging from linear scalarization to complex non-linear logical operators, which are often handled separately in previous literature. We prove that our method exactly recovers the target distribution for linear scalarization and quantify the approximation quality for nonlinear operators through a distortion factor. Experiments on a synthetic 2D grid and real-world molecule-generation tasks demonstrate that our approach achieves performance comparable to baselines that require additional training.