Factual and Personalized Recommendations using Language Models and Reinforcement Learning

TL;DR

This paper introduces P4LM, a language model that combines factual grounding and reinforcement learning to provide personalized, precise, and explainable recommendations in conversational systems, demonstrated on movie recommendations.

Contribution

The work presents a novel language model architecture that integrates factual grounding and reinforcement learning for personalized recommendations with explanations.

Findings

P4LM outperforms baseline models in personalization and relevance.

Reinforcement learning improves the factual accuracy of recommendations.

The model effectively generates engaging and tailored movie narratives.

Abstract

Recommender systems (RSs) play a central role in connecting users to content, products, and services, matching candidate items to users based on their preferences. While traditional RSs rely on implicit user feedback signals, conversational RSs interact with users in natural language. In this work, we develop a comPelling, Precise, Personalized, Preference-relevant language model (P4LM) that recommends items to users while putting emphasis on explaining item characteristics and their relevance. P4LM uses the embedding space representation of a user's preferences to generate compelling responses that are factually-grounded and relevant w.r.t. the user's preferences. Moreover, we develop a joint reward function that measures precision, appeal, and personalization, which we use as AI-based feedback in a reinforcement learning-based language model framework. Using the MovieLens 25M dataset, we demonstrate that P4LM delivers compelling, personalized movie narratives to users.