OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios

TL;DR

This paper introduces OmniChat, a scalable synthetic data-driven spoken dialogue system that improves handling diverse real-world scenarios, including audio and emotional expressions, by leveraging the new ShareChatX dataset.

Contribution

The paper presents ShareChatX, a large-scale diverse dialogue dataset, and OmniChat, a multi-turn dialogue system optimized with synthetic data for complex scenarios.

Findings

Synthetic data enhances dialogue system performance in complex scenarios.

Optimal synthetic-to-real data ratio improves real-world task results.

OmniChat achieves state-of-the-art results on DailyTalk dataset.

Abstract

With the rapid development of large language models, researchers have created increasingly advanced spoken dialogue systems that can naturally converse with humans. However, these systems still struggle to handle the full complexity of real-world conversations, including audio events, musical contexts, and emotional expressions, mainly because current dialogue datasets are constrained in both scale and scenario diversity. In this paper, we propose leveraging synthetic data to enhance the dialogue models across diverse scenarios. We introduce ShareChatX, the first comprehensive, large-scale dataset for spoken dialogue that spans diverse scenarios. Based on this dataset, we introduce OmniChat, a multi-turn dialogue system with a heterogeneous feature fusion module, designed to optimize feature selection in different dialogue contexts. In addition, we explored critical aspects of training dialogue systems using synthetic data. Through comprehensive experimentation, we determined the ideal balance between synthetic and real data, achieving state-of-the-art results on the real-world dialogue dataset DailyTalk. We also highlight the crucial importance of synthetic data in tackling diverse, complex dialogue scenarios, especially those involving audio and music. For more details, please visit our demo page at \url{https://sharechatx.github.io/}.