LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue

TL;DR

This paper introduces a new theoretical framework for dialogue text-generation, addressing issues like lexical diversity and task success, and demonstrates its practical benefits through improved algorithms and predictive statistics.

Contribution

The paper presents a novel theoretical framework for understanding and improving dialogue text-generation, with guarantees for unseen data and insights into data-shift effects.

Findings

Improved task-success and human-likeness in generated dialogue.

Theoretical statistics predict dialogue quality.

Enhanced understanding of data-shift in dialogue models.

Abstract

Algorithms for text-generation in dialogue can be misguided. For example, in task-oriented settings, reinforcement learning that optimizes only task-success can lead to abysmal lexical diversity. We hypothesize this is due to poor theoretical understanding of the objectives in text-generation and their relation to the learning process (i.e., model training). To this end, we propose a new theoretical framework for learning to generate text in dialogue. Compared to existing theories of learning, our framework allows for analysis of the multi-faceted goals inherent to text-generation. We use our framework to develop theoretical guarantees for learners that adapt to unseen data. As an example, we apply our theory to study data-shift within a cooperative learning algorithm proposed for the GuessWhat?! visual dialogue game. From this insight, we propose a new algorithm, and empirically, we demonstrate our proposal improves both task-success and human-likeness of the generated text. Finally, we show statistics from our theory are empirically predictive of multiple qualities of the generated dialogue, suggesting our theory is useful for model-selection when human evaluations are not available.