It Couldn't Help But Overhear: On the Limits of Modelling Meta-Communicative Grounding Acts with Supervised Learning

TL;DR

This paper critically examines the limitations of using supervised learning to model meta-communicative grounding acts in dialogue, highlighting the challenges faced by overhearing models in capturing human-like understanding.

Contribution

It presents a preliminary analysis showing the inherent difficulties of modeling human meta-communicative acts with data-driven approaches and discusses the implications for dialogue systems.

Findings

Evidence of variability in human clarification requests

Supervised models struggle to replicate human grounding acts

Discussion on the limitations of overhearing paradigms in NLP

Abstract

Active participation in a conversation is key to building common ground, since understanding is jointly tailored by producers and recipients. Overhearers are deprived of the privilege of performing grounding acts and can only conjecture about intended meanings. Still, data generation and annotation, modelling, training and evaluation of NLP dialogue models place reliance on the overhearing paradigm. How much of the underlying grounding processes are thereby forfeited? As we show, there is evidence pointing to the impossibility of properly modelling human meta-communicative acts with data-driven learning models. In this paper, we discuss this issue and provide a preliminary analysis on the variability of human decisions for requesting clarification. Most importantly, we wish to bring this topic back to the community's table, encouraging discussion on the consequences of having models designed to only "listen in".