NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions

TL;DR

This paper introduces NUANCED, a new dataset and approach for user-centric conversational systems that model user preferences as distributions over system ontologies, enabling more natural and flexible interactions.

Contribution

The work presents a novel dataset NUANCED and a framework for mapping free-form user utterances to estimated preference distributions, advancing user-centric dialogue modeling.

Findings

NUANCED contains 5.1k dialogues and 26k turns.

Experiments show the feasibility of user preference distribution modeling.

Challenges remain in reasoning over diverse knowledge types.

Abstract

Existing conversational systems are mostly agent-centric, which assumes the user utterances would closely follow the system ontology (for NLU or dialogue state tracking). However, in real-world scenarios, it is highly desirable that the users can speak freely in their own way. It is extremely hard, if not impossible, for the users to adapt to the unknown system ontology. In this work, we attempt to build a user-centric dialogue system. As there is no clean mapping for a user's free form utterance to an ontology, we first model the user preferences as estimated distributions over the system ontology and map the users' utterances to such distributions. Learning such a mapping poses new challenges on reasoning over existing knowledge, ranging from factoid knowledge, commonsense knowledge to the users' own situations. To this end, we build a new dataset named NUANCED that focuses on such realistic settings for conversational recommendation. Collected via dialogue simulation and paraphrasing, NUANCED contains 5.1k dialogues, 26k turns of high-quality user responses. We conduct experiments, showing both the usefulness and challenges of our problem setting. We believe NUANCED can serve as a valuable resource to push existing research from the agent-centric system to the user-centric system. The code and data is publicly available at \url{https://github.com/facebookresearch/nuanced}.