A Large-Scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search

TL;DR

This paper conducts a large-scale analysis of over 150,000 dialogue transcripts to understand the structural properties of information-seeking conversations, revealing insights that relate conversational search to other AI tasks and identifying dataset limitations.

Contribution

It provides a comprehensive analysis of mixed initiative patterns in dialogue datasets, linking conversational search to broader AI research and highlighting gaps in existing data for future collection.

Findings

Identifies common mixed initiative patterns in information-seeking dialogues.

Highlights similarities between conversational search and professional librarian interviews.

Uncovers limitations in current datasets for training conversational AI.

Abstract

Conversational search is a relatively young area of research that aims at automating an information-seeking dialogue. In this paper we help to position it with respect to other research areas within conversational Artificial Intelligence (AI) by analysing the structural properties of an information-seeking dialogue. To this end, we perform a large-scale dialogue analysis of more than 150K transcripts from 16 publicly available dialogue datasets. These datasets were collected to inform different dialogue-based tasks including conversational search. We extract different patterns of mixed initiative from these dialogue transcripts and use them to compare dialogues of different types. Moreover, we contrast the patterns found in information-seeking dialogues that are being used for research purposes with the patterns found in virtual reference interviews that were conducted by professional librarians. The insights we provide (1) establish close relations between conversational search and other conversational AI tasks; and (2) uncover limitations of existing conversational datasets to inform future data collection tasks.