Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems

TL;DR

This paper introduces a novel method for generating usage-related questions in conversational recommender systems, utilizing a new dataset and models to improve preference elicitation through natural language questions about item usage.

Contribution

It presents a multi-stage crowdsourcing annotation protocol and develops four question generation models, advancing preference elicitation in conversational recommender systems.

Findings

Neural models outperform template-based models in question quality.

Automatic and human evaluations show effective question generation.

Analysis reveals limitations and areas for future improvement.

Abstract

A key distinguishing feature of conversational recommender systems over traditional recommender systems is their ability to elicit user preferences using natural language. Currently, the predominant approach to preference elicitation is to ask questions directly about items or item attributes. Users searching for recommendations may not have deep knowledge of the available options in a given domain. As such, they might not be aware of key attributes or desirable values for them. However, in many settings, talking about the planned use of items does not present any difficulties, even for those that are new to a domain. In this paper, we propose a novel approach to preference elicitation by asking implicit questions based on item usage. As one of the main contributions of this work, we develop a multi-stage data annotation protocol using crowdsourcing, to create a high-quality labeled training dataset. Another main contribution is the development of four models for the question generation task: two template-based baseline models and two neural text-to-text models. The template-based models use heuristically extracted common patterns found in the training data, while the neural models use the training data to learn to generate questions automatically. Using common metrics from machine translation for automatic evaluation, we show that our approaches are effective in generating elicitation questions, even with limited training data. We further employ human evaluation for comparing the generated questions using both pointwise and pairwise evaluation designs. We find that the human evaluation results are consistent with the automatic ones, allowing us to draw conclusions about the quality of the generated questions with certainty. Finally, we provide a detailed analysis of cases where the models show their limitations.