Building the Intent Landscape of Real-World Conversational Corpora with Extractive Question-Answering Transformers

TL;DR

This paper introduces an unsupervised pipeline using extractive QA transformers and clustering to map and understand intents in noisy real-world conversational data, aiding natural language understanding applications.

Contribution

It presents a novel unsupervised method combining extractive QA and clustering to extract and organize intents from real-world dialogues, even in noisy data environments.

Findings

Achieved over 85% linguistic validation rate for intent spans.

Reconstructed intent schemes with an average recall of 94.3%.

Demonstrated generalization of ELECTRA model to dialogue understanding.

Abstract

For companies with customer service, mapping intents inside their conversational data is crucial in building applications based on natural language understanding (NLU). Nevertheless, there is no established automated technique to gather the intents from noisy online chats or voice transcripts. Simple clustering approaches are not suited to intent-sparse dialogues. To solve this intent-landscape task, we propose an unsupervised pipeline that extracts the intents and the taxonomy of intents from real-world dialogues. Our pipeline mines intent-span candidates with an extractive Question-Answering Electra model and leverages sentence embeddings to apply a low-level density clustering followed by a top-level hierarchical clustering. Our results demonstrate the generalization ability of an ELECTRA large model fine-tuned on the SQuAD2 dataset to understand dialogues. With the right prompting question, this model achieves a rate of linguistic validation on intent spans beyond 85%. We furthermore reconstructed the intent schemes of five domains from the MultiDoGo dataset with an average recall of 94.3%.