Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems

TL;DR

This paper introduces ChatCRS, a framework that enhances large language models in conversational recommender systems by integrating external knowledge and goal guidance, significantly improving recommendation accuracy and dialogue quality.

Contribution

The paper proposes a novel ChatCRS framework with knowledge retrieval and goal planning agents, enabling LLMs to better utilize external knowledge and guide conversations effectively.

Findings

Achieves state-of-the-art performance on multi-goal CRS datasets.

Improves informativeness of responses by 17%.

Enhances proactivity in dialogues by 27%.

Abstract

This paper aims to efficiently enable large language models (LLMs) to use external knowledge and goal guidance in conversational recommender system (CRS) tasks. Advanced LLMs (e.g., ChatGPT) are limited in domain-specific CRS tasks for 1) generating grounded responses with recommendation-oriented knowledge, or 2) proactively leading the conversations through different dialogue goals. In this work, we first analyze those limitations through a comprehensive evaluation, showing the necessity of external knowledge and goal guidance which contribute significantly to the recommendation accuracy and language quality. In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into several sub-tasks through the implementation of 1) a knowledge retrieval agent using a tool-augmented approach to reason over external Knowledge Bases and 2) a goal-planning agent for dialogue goal prediction. Experimental results on two multi-goal CRS datasets reveal that ChatCRS sets new state-of-the-art benchmarks, improving language quality of informativeness by 17% and proactivity by 27%, and achieving a tenfold enhancement in recommendation accuracy.