LLM-Enhanced Reinforcement Learning for Long-Term User Satisfaction in Interactive Recommendation

TL;DR

This paper introduces LERL, a hierarchical recommendation framework combining LLM-based semantic planning with reinforcement learning to enhance long-term user satisfaction and content diversity.

Contribution

It proposes a novel hierarchical approach that integrates large language models with reinforcement learning for improved long-term recommendation performance.

Findings

LERL significantly outperforms state-of-the-art baselines in long-term user satisfaction.

The hierarchical design reduces action space and improves planning efficiency.

Experiments on real-world datasets validate the effectiveness of LERL.

Abstract

Interactive recommender systems can dynamically adapt to user feedback, but often suffer from content homogeneity and filter bubble effects due to overfitting short-term user preferences. While recent efforts aim to improve content diversity, they predominantly operate in static or one-shot settings, neglecting the long-term evolution of user interests. Reinforcement learning provides a principled framework for optimizing long-term user satisfaction by modeling sequential decision-making processes. However, its application in recommendation is hindered by sparse, long-tailed user-item interactions and limited semantic planning capabilities. In this work, we propose LLM-Enhanced Reinforcement Learning (LERL), a novel hierarchical recommendation framework that integrates the semantic planning power of LLM with the fine-grained adaptability of RL. LERL consists of a high-level LLM-based planner that selects semantically diverse content categories, and a low-level RL policy that recommends personalized items within the selected semantic space. This hierarchical design narrows the action space, enhances planning efficiency, and mitigates overexposure to redundant content. Extensive experiments on real-world datasets demonstrate that LERL significantly improves long-term user satisfaction when compared with state-of-the-art baselines. The implementation of LERL is available at https://github.com/1163710212/LERL.