An Empirical Study of Topic Transition in Dialogue

TL;DR

This paper empirically investigates how humans transition between topics in conversations, analyzing 215 dialogues and modeling these transitions to improve open-domain dialog systems.

Contribution

It provides the first detailed annotation and analysis of topic transitions in the switchboard corpus, and develops a model achieving over 82% precision in predicting topic shifts.

Findings

Analyzed 215 conversations for topic transition patterns.

Developed a model with 83% in-domain and 82% out-of-domain precision.

Insights aim to enhance human-like topic transitions in dialog systems.

Abstract

Transitioning between topics is a natural component of human-human dialog. Although topic transition has been studied in dialogue for decades, only a handful of corpora based studies have been performed to investigate the subtleties of topic transitions. Thus, this study annotates 215 conversations from the switchboard corpus and investigates how variables such as length, number of topic transitions, topic transitions share by participants and turns/topic are related. This work presents an empirical study on topic transition in switchboard corpus followed by modelling topic transition with a precision of 83% for in-domain(id) test set and 82% on 10 out-of-domain}(ood) test set. It is envisioned that this work will help in emulating human-human like topic transition in open-domain dialog systems.