We've had this conversation before: A Novel Approach to Measuring Dialog Similarity

TL;DR

This paper introduces a novel dialog similarity measure based on an adapted edit distance that considers semantics, flow, and participants, outperforming existing methods in aligning with human perception.

Contribution

The paper presents a new dialog similarity metric that incorporates conversation semantics, flow, and participant roles, improving upon existing document similarity measures.

Findings

Outperforms existing similarity measures in capturing dialog flow

Aligns better with human perception of conversation similarity

Effective on publicly available datasets

Abstract

Dialog is a core building block of human natural language interactions. It contains multi-party utterances used to convey information from one party to another in a dynamic and evolving manner. The ability to compare dialogs is beneficial in many real world use cases, such as conversation analytics for contact center calls and virtual agent design. We propose a novel adaptation of the edit distance metric to the scenario of dialog similarity. Our approach takes into account various conversation aspects such as utterance semantics, conversation flow, and the participants. We evaluate this new approach and compare it to existing document similarity measures on two publicly available datasets. The results demonstrate that our method outperforms the other approaches in capturing dialog flow, and is better aligned with the human perception of conversation similarity.