GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks

TL;DR

This paper introduces GFlowGR, a novel fine-tuning framework for generative recommendation systems that uses Generative Flow Networks to address exposure bias and improve recommendation quality.

Contribution

It proposes a GFlowNets-based fine-tuning method that incorporates collaborative knowledge and diverse sampling to mitigate exposure bias in generative recommendation models.

Findings

GFlowGR outperforms traditional fine-tuning methods on real-world datasets.

The framework effectively reduces exposure bias in generative recommendation systems.

GFlowGR demonstrates robustness across different datasets and backbone models.

Abstract

Generative recommendations (GR), which usually include item tokenizers and generative Large Language Models (LLMs), have demonstrated remarkable success across a wide range of scenarios. The majority of existing research efforts primarily concentrate on developing powerful item tokenizers or advancing LLM decoding strategies to attain superior performance. However, the critical fine-tuning step in GR frameworks, which is essential for adapting LLMs to recommendation data, remains largely unexplored. Current approaches predominantly rely on either the next-token prediction loss of supervised fine-tuning (SFT) or recommendationspecific direct preference optimization (DPO) strategies. Both methods ignore the exploration of possible positive unobserved samples, which is commonly referred to as the exposure bias problem. To mitigate this problem, this paper treats the GR as a multi-step generation task and constructs a GFlowNets-based fine-tuning framework (GFlowGR). The proposed framework integrates collaborative knowledge from traditional recommender systems to create an adaptive trajectory sampler and a comprehensive reward model. Leveraging the diverse generation property of GFlowNets, along with sampling and heuristic weighting techniques, GFlowGR emerges as a promising approach to mitigate the exposure bias problem. Extensive empirical results on two real-world datasets and with two different GR backbones highlight the effectiveness and robustness of GFlowGR.