Frame of Reference: Addressing the Challenges of Common Ground Representation in Situational Dialogs

TL;DR

This paper evaluates how well language models establish and utilize common ground in situated dialogs, especially with relational references, and proposes reinforcement learning methods to enhance this capability.

Contribution

It introduces methods for representing common ground in situated dialogs and demonstrates improvements using reinforcement learning on synthetic data.

Findings

Models can perform basic grounding acts like acknowledgments.

Reinforcement learning improves the handling of relational references.

Proposed approaches enhance common ground representation in dialog systems.

Abstract

Common ground plays a critical role in situated spoken dialogs, where interlocutors must establish and maintain shared references to entities, events, and relations to sustain coherent interaction in a shared space and over time. With the increasing presence of embodied conversational agents and social robots, the ability to correctly ground this kind of conversational content in order to refer back later also becomes important for dialog systems. Prior studies have demonstrated that LLMs are capable of performing certain grounding acts like acknowledgments. However, relatively little work has investigated their capacity to leverage the grounded information, like in complex scenarios involving space and time (e.g., "let's go to that caf\'e near the park we went to yesterday"). To that end, in this work, we evaluate a model's ability to establish common ground by utilizing these "relational references" in the dynamic and shared environments of situated dialogs. We then test multiple methods for representing common ground and further propose approaches to improve their performance by using reinforcement learning on our synthetically generated dialog data .