Def-DTS: Deductive Reasoning for Open-domain Dialogue Topic Segmentation

TL;DR

This paper introduces Def-DTS, a novel approach using large language model-based deductive reasoning to improve open-domain dialogue topic segmentation, addressing data scarcity and ambiguity issues.

Contribution

It presents a structured prompting method leveraging LLMs for multi-step reasoning in DTS, including a domain-agnostic intent classification and case study analysis.

Findings

Def-DTS outperforms existing methods in various dialogue settings.

The approach reduces type 2 errors in topic shift detection.

Autolabeling with LLM reasoning shows promising potential.

Abstract

Dialogue Topic Segmentation (DTS) aims to divide dialogues into coherent segments. DTS plays a crucial role in various NLP downstream tasks, but suffers from chronic problems: data shortage, labeling ambiguity, and incremental complexity of recently proposed solutions. On the other hand, Despite advances in Large Language Models (LLMs) and reasoning strategies, these have rarely been applied to DTS. This paper introduces Def-DTS: Deductive Reasoning for Open-domain Dialogue Topic Segmentation, which utilizes LLM-based multi-step deductive reasoning to enhance DTS performance and enable case study using intermediate result. Our method employs a structured prompting approach for bidirectional context summarization, utterance intent classification, and deductive topic shift detection. In the intent classification process, we propose the generalizable intent list for domain-agnostic dialogue intent classification. Experiments in various dialogue settings demonstrate that Def-DTS consistently outperforms traditional and state-of-the-art approaches, with each subtask contributing to improved performance, particularly in reducing type 2 error. We also explore the potential for autolabeling, emphasizing the importance of LLM reasoning techniques in DTS.