Learning to Generate Equitable Text in Dialogue from Biased Training Data

TL;DR

This paper formalizes the concept of equity in dialogue text generation, establishes theoretical links between human-likeness and equity, and empirically tests these ideas using algorithms in a visual dialogue task.

Contribution

It introduces formal definitions of equity in dialogue, connects learning equity with human-likeness, and demonstrates how equitable text can be learned from biased data without modifications.

Findings

Theoretical connection between learning human-likeness and equity.

Algorithms can learn equitable text without data modifications under certain conditions.

Empirical validation in the GuessWhat?! dialogue game supports the theory.

Abstract

The ingrained principles of fairness in a dialogue system's decision-making process and generated responses are crucial for user engagement, satisfaction, and task achievement. Absence of equitable and inclusive principles can hinder the formation of common ground, which in turn negatively impacts the overall performance of the system. For example, misusing pronouns in a user interaction may cause ambiguity about the intended subject. Yet, there is no comprehensive study of equitable text generation in dialogue. Aptly, in this work, we use theories of computational learning to study this problem. We provide formal definitions of equity in text generation, and further, prove formal connections between learning human-likeness and learning equity: algorithms for improving equity ultimately reduce to algorithms for improving human-likeness (on augmented data). With this insight, we also formulate reasonable conditions under which text generation algorithms can learn to generate equitable text without any modifications to the biased training data on which they learn. To exemplify our theory in practice, we look at a group of algorithms for the GuessWhat?! visual dialogue game and, using this example, test our theory empirically. Our theory accurately predicts relative-performance of multiple algorithms in generating equitable text as measured by both human and automated evaluation.