Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers

TL;DR

This paper introduces a training-free, LLM-based explore-exploit method for generative recommender systems that improves user engagement by balancing exploration of new content and exploitation of known preferences, demonstrated through CTR improvements.

Contribution

It presents a novel training-free optimization approach connecting user feedback to LLMs, enabling dynamic recommendation improvements without costly fine-tuning.

Findings

The approach consistently increases Click Through Rate (CTR).

Generative exploration helps discover hidden user preferences.

Ablation shows exploration is crucial for learning preferences.

Abstract

Recommender systems are widely used to suggest engaging content, and Large Language Models (LLMs) have given rise to generative recommenders. Such systems can directly generate items, including for open-set tasks like question suggestion. While the world knowledge of LLMs enable good recommendations, improving the generated content through user feedback is challenging as continuously fine-tuning LLMs is prohibitively expensive. We present a training-free approach for optimizing generative recommenders by connecting user feedback loops to LLM-based optimizers. We propose a generative explore-exploit method that can not only exploit generated items with known high engagement, but also actively explore and discover hidden population preferences to improve recommendation quality. We evaluate our approach on question generation in two domains (e-commerce and general knowledge), and model user feedback with Click Through Rate (CTR). Experiments show our LLM-based explore-exploit approach can iteratively improve recommendations, and consistently increase CTR. Ablation analysis shows that generative exploration is key to learning user preferences, avoiding the pitfalls of greedy exploit-only approaches. A human evaluation strongly supports our quantitative findings.