DeliData: A dataset for deliberation in multi-party problem solving

TL;DR

This paper introduces DeliData, the first public dataset of group deliberation conversations on a cognitive task, highlighting its potential for studying collaborative problem solving and developing dialogue generation methods.

Contribution

It provides a novel dataset with annotations capturing deliberation cues, enabling research on collaborative dialogue and problem solving.

Findings

64% of groups found better solutions through deliberation

43.8% of groups with correct final answers had no correct individual solutions

Developed and evaluated two methods for generating deliberation utterances

Abstract

Group deliberation enables people to collaborate and solve problems, however, it is understudied due to a lack of resources. To this end, we introduce the first publicly available dataset containing collaborative conversations on solving a well-established cognitive task, consisting of 500 group dialogues and 14k utterances. In 64% of these conversations, the group members are able to find a better solution than they had identified individually, and in 43.8% of the groups who had a correct answer as their final solution, none of the participants had solved the task correctly by themselves. Furthermore, we propose a novel annotation schema that captures deliberation cues and release all 14k utterances annotated with it. Finally, we use the proposed dataset to develop and evaluate two methods for generating deliberation utterances. The data collection platform, dataset and annotated corpus are publicly available at https://delibot.xyz.