Overprotective Training Environments Fall Short at Testing Time: Let Models Contribute to Their Own Training

TL;DR

This paper addresses the discrepancy between training and testing conditions in conversational systems by training models with mixed human and machine-generated dialogues, improving their robustness in real-world interactions.

Contribution

It introduces a novel training method that incorporates both human and machine dialogues to better prepare models for real-world testing scenarios.

Findings

Improved performance on GuessWhat?! visual referential game

Models become more robust to noisy, real-world data

Training with mixed dialogues enhances conversational naturalness

Abstract

Despite important progress, conversational systems often generate dialogues that sound unnatural to humans. We conjecture that the reason lies in their different training and testing conditions: agents are trained in a controlled "lab" setting but tested in the "wild". During training, they learn to generate an utterance given the human dialogue history. On the other hand, during testing, they must interact with each other, and hence deal with noisy data. We propose to fill this gap by training the model with mixed batches containing both samples of human and machine-generated dialogues. We assess the validity of the proposed method on GuessWhat?!, a visual referential game.