Beyond Retrieval: Generating Narratives in Conversational Recommender Systems

TL;DR

This paper introduces a new dataset and a fusion architecture to enhance conversational recommender systems with narrative generation, leveraging large language models for personalized explanations and summaries.

Contribution

It presents the REGEN dataset with rich user narratives and a fusion model combining collaborative filtering and LLMs for improved narrative generation in recommendations.

Findings

LLMs effectively learn from simple fusion architectures with CF embeddings

Combining CF and content embeddings improves language metrics by 4-12%

Analysis shows how CF and content embeddings contribute to narrative generation

Abstract

The recent advances in Large Language Model's generation and reasoning capabilities present an opportunity to develop truly conversational recommendation systems. However, effectively integrating recommender system knowledge into LLMs for natural language generation which is tailored towards recommendation tasks remains a challenge. This paper addresses this challenge by making two key contributions. First, we introduce a new dataset (REGEN) for natural language generation tasks in conversational recommendations. REGEN (Reviews Enhanced with GEnerative Narratives) extends the Amazon Product Reviews dataset with rich user narratives, including personalized explanations of product preferences, product endorsements for recommended items, and summaries of user purchase history. REGEN is made publicly available to facilitate further research. Furthermore, we establish benchmarks using well-known generative metrics, and perform an automated evaluation of the new dataset using a rater LLM. Second, the paper introduces a fusion architecture (CF model with an LLM) which serves as a baseline for REGEN. And to the best of our knowledge, represents the first attempt to analyze the capabilities of LLMs in understanding recommender signals and generating rich narratives. We demonstrate that LLMs can effectively learn from simple fusion architectures utilizing interaction-based CF embeddings, and this can be further enhanced using the metadata and personalization data associated with items. Our experiments show that combining CF and content embeddings leads to improvements of 4-12% in key language metrics compared to using either type of embedding individually. We also provide an analysis to interpret how CF and content embeddings contribute to this new generative task.