Conversational Entity Linking: Problem Definition and Datasets

TL;DR

This paper defines conversational entity linking, analyzes dialogue datasets to identify its unique challenges, and evaluates traditional EL systems on a new dataset, providing resources for further research.

Contribution

It introduces the ConEL dataset, characterizes conversational EL, and extends existing EL methods to better suit conversational contexts.

Findings

Traditional EL systems perform poorly on conversational data

Annotated datasets reveal unique challenges of conversational entity linking

Extended methods improve EL performance in conversations

Abstract

Machine understanding of user utterances in conversational systems is of utmost importance for enabling engaging and meaningful conversations with users. Entity Linking (EL) is one of the means of text understanding, with proven efficacy for various downstream tasks in information retrieval. In this paper, we study entity linking for conversational systems. To develop a better understanding of what EL in a conversational setting entails, we analyze a large number of dialogues from existing conversational datasets and annotate references to concepts, named entities, and personal entities using crowdsourcing. Based on the annotated dialogues, we identify the main characteristics of conversational entity linking. Further, we report on the performance of traditional EL systems on our Conversational Entity Linking dataset, ConEL, and present an extension to these methods to better fit the conversational setting. The resources released with this paper include annotated datasets, detailed descriptions of crowdsourcing setups, as well as the annotations produced by various EL systems. These new resources allow for an investigation of how the role of entities in conversations is different from that in documents or isolated short text utterances like queries and tweets, and complement existing conversational datasets.