Asking Clarifying Questions Based on Negative Feedback in Conversational Search

TL;DR

This paper introduces a new intent clarification task in conversational search, where a BERT-based model uses negative feedback and MMR principles to ask effective yes/no questions, improving user intent identification and document retrieval efficiency.

Contribution

It proposes a novel intent clarification task utilizing negative feedback and MMR-BERT to select optimal questions, advancing conversational search capabilities.

Findings

MMR-BERT outperforms state-of-the-art baselines significantly.

Selected questions improve document retrieval performance.

The approach reduces the number of conversation turns needed.

Abstract

Users often need to look through multiple search result pages or reformulate queries when they have complex information-seeking needs. Conversational search systems make it possible to improve user satisfaction by asking questions to clarify users' search intents. This, however, can take significant effort to answer a series of questions starting with "what/why/how". To quickly identify user intent and reduce effort during interactions, we propose an intent clarification task based on yes/no questions where the system needs to ask the correct question about intents within the fewest conversation turns. In this task, it is essential to use negative feedback about the previous questions in the conversation history. To this end, we propose a Maximum-Marginal-Relevance (MMR) based BERT model (MMR-BERT) to leverage negative feedback based on the MMR principle for the next clarifying question selection. Experiments on the Qulac dataset show that MMR-BERT outperforms state-of-the-art baselines significantly on the intent identification task and the selected questions also achieve significantly better performance in the associated document retrieval tasks.