An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues

TL;DR

This paper introduces ConversationShape, an unsupervised metric to analyze mixed-initiative dialogues, revealing how participant roles influence dialogue quality in conversational search systems.

Contribution

It proposes ConversationShape as a novel unsupervised metric for analyzing dialogue roles and compares various datasets to understand mixed-initiative interactions.

Findings

Deviations in ConversationShape predict dialogue quality.

Different datasets exhibit distinct ConversationShape patterns.

Insights into human-human vs. human-machine dialogues.

Abstract

The ability to engage in mixed-initiative interaction is one of the core requirements for a conversational search system. How to achieve this is poorly understood. We propose a set of unsupervised metrics, termed ConversationShape, that highlights the role each of the conversation participants plays by comparing the distribution of vocabulary and utterance types. Using ConversationShape as a lens, we take a closer look at several conversational search datasets and compare them with other dialogue datasets to better understand the types of dialogue interaction they represent, either driven by the information seeker or the assistant. We discover that deviations from the ConversationShape of a human-human dialogue of the same type is predictive of the quality of a human-machine dialogue.