HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data

TL;DR

HybriDialogue is a new dataset of multi-turn dialogues grounded on both Wikipedia text and tables, designed to improve systems' ability to reason over mixed modalities in information-seeking conversations.

Contribution

The paper introduces HybriDialogue, a novel dataset combining text and tables for multi-turn dialogues, along with baseline tasks and experiments to advance multimodal dialogue systems.

Findings

Baseline results indicate significant room for improvement.

The dataset enables research on reasoning over combined textual and tabular data.

It highlights the challenges of multi-modal information grounding in dialogue systems.

Abstract

A pressing challenge in current dialogue systems is to successfully converse with users on topics with information distributed across different modalities. Previous work in multiturn dialogue systems has primarily focused on either text or table information. In more realistic scenarios, having a joint understanding of both is critical as knowledge is typically distributed over both unstructured and structured forms. We present a new dialogue dataset, HybriDialogue, which consists of crowdsourced natural conversations grounded on both Wikipedia text and tables. The conversations are created through the decomposition of complex multihop questions into simple, realistic multiturn dialogue interactions. We propose retrieval, system state tracking, and dialogue response generation tasks for our dataset and conduct baseline experiments for each. Our results show that there is still ample opportunity for improvement, demonstrating the importance of building stronger dialogue systems that can reason over the complex setting of information-seeking dialogue grounded on tables and text.