# The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue

## TL;DR

The paper presents the PhotoBook dataset, a large collection of visually-grounded, task-oriented dialogues designed to study shared understanding and reference resolution in conversation, along with a baseline model demonstrating the importance of shared context.

## Contribution

It introduces the PhotoBook dataset with detailed dialogue data and analysis, and proposes a baseline model highlighting the role of shared information in reference resolution.

## Key findings

- Shared information is crucial for resolving references in later dialogue turns.
- The dataset enables studying common ground development in dialogue.
- Baseline model shows the importance of shared context for reference resolution.

## Abstract

This paper introduces the PhotoBook dataset, a large-scale collection of visually-grounded, task-oriented dialogues in English designed to investigate shared dialogue history accumulating during conversation. Taking inspiration from seminal work on dialogue analysis, we propose a data-collection task formulated as a collaborative game prompting two online participants to refer to images utilising both their visual context as well as previously established referring expressions. We provide a detailed description of the task setup and a thorough analysis of the 2,500 dialogues collected. To further illustrate the novel features of the dataset, we propose a baseline model for reference resolution which uses a simple method to take into account shared information accumulated in a reference chain. Our results show that this information is particularly important to resolve later descriptions and underline the need to develop more sophisticated models of common ground in dialogue interaction.