The JDDC 2.0 Corpus: A Large-Scale Multimodal Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service

TL;DR

The paper introduces JDDC 2.0, a large-scale multimodal Chinese dialogue dataset for e-commerce customer service, facilitating research on understanding and generating multi-turn dialogues involving text and images.

Contribution

It provides a comprehensive, annotated dataset with millions of utterances and images, and shares solutions from top teams in a multimodal dialogue challenge.

Findings

Dataset contains 246,000 dialogue sessions and 3 million utterances.

Includes product knowledge bases and image category annotations.

Provides insights from top challenge solutions.

Abstract

With the development of the Internet, more and more people get accustomed to online shopping. When communicating with customer service, users may express their requirements by means of text, images, and videos, which precipitates the need for understanding these multimodal information for automatic customer service systems. Images usually act as discriminators for product models, or indicators of product failures, which play important roles in the E-commerce scenario. On the other hand, detailed information provided by the images is limited, and typically, customer service systems cannot understand the intents of users without the input text. Thus, bridging the gap of the image and text is crucial for the multimodal dialogue task. To handle this problem, we construct JDDC 2.0, a large-scale multimodal multi-turn dialogue dataset collected from a mainstream Chinese E-commerce platform (JD.com), containing about 246 thousand dialogue sessions, 3 million utterances, and 507 thousand images, along with product knowledge bases and image category annotations. We present the solutions of top-5 teams participating in the JDDC multimodal dialogue challenge based on this dataset, which provides valuable insights for further researches on the multimodal dialogue task.