Measuring Semantic Coherence of a Conversation

TL;DR

This paper introduces methods to measure the semantic coherence of conversations using knowledge graphs and embeddings, aiming to improve understanding of conversational structure and semantics.

Contribution

It proposes novel graph-based and machine learning approaches for assessing semantic coherence in conversations, leveraging knowledge graphs and embeddings.

Findings

Effective detection of coherence patterns in Ubuntu Dialogue Corpus

Graph-based methods outperform baseline models

Machine learning models capture nuanced semantic relations

Abstract

Conversational systems have become increasingly popular as a way for humans to interact with computers. To be able to provide intelligent responses, conversational systems must correctly model the structure and semantics of a conversation. We introduce the task of measuring semantic (in)coherence in a conversation with respect to background knowledge, which relies on the identification of semantic relations between concepts introduced during a conversation. We propose and evaluate graph-based and machine learning-based approaches for measuring semantic coherence using knowledge graphs, their vector space embeddings and word embedding models, as sources of background knowledge. We demonstrate how these approaches are able to uncover different coherence patterns in conversations on the Ubuntu Dialogue Corpus.