Increasing faithfulness in human-human dialog summarization with Spoken Language Understanding tasks

TL;DR

This paper explores enhancing human-human dialogue summarization by integrating Spoken Language Understanding tasks to improve semantic faithfulness, introducing new evaluation metrics and dataset enhancements, and demonstrating improved accuracy on call center dialogues.

Contribution

It introduces a novel approach combining SLU tasks with dialogue summarization, new semantic evaluation criteria, and an augmented dataset for better task-oriented dialogue summarization.

Findings

Incorporating task-related information improves summary accuracy.

Semantic evaluation criteria better reflect summary faithfulness.

Enhanced dataset supports more effective research.

Abstract

Dialogue summarization aims to provide a concise and coherent summary of conversations between multiple speakers. While recent advancements in language models have enhanced this process, summarizing dialogues accurately and faithfully remains challenging due to the need to understand speaker interactions and capture relevant information. Indeed, abstractive models used for dialog summarization may generate summaries that contain inconsistencies. We suggest using the semantic information proposed for performing Spoken Language Understanding (SLU) in human-machine dialogue systems for goal-oriented human-human dialogues to obtain a more semantically faithful summary regarding the task. This study introduces three key contributions: First, we propose an exploration of how incorporating task-related information can enhance the summarization process, leading to more semantically accurate summaries. Then, we introduce a new evaluation criterion based on task semantics. Finally, we propose a new dataset version with increased annotated data standardized for research on task-oriented dialogue summarization. The study evaluates these methods using the DECODA corpus, a collection of French spoken dialogues from a call center. Results show that integrating models with task-related information improves summary accuracy, even with varying word error rates.