Calibrate your listeners! Robust communication-based training for pragmatic speakers

TL;DR

This paper introduces a population-based training method for pragmatic speakers in NLP, using calibrated neural listeners to reduce semantic drift and improve contextually useful utterances.

Contribution

It proposes a novel ensemble-based listener calibration approach that enhances communication training for NLP systems, addressing semantic drift issues.

Findings

Ensemble listeners provide better uncertainty calibration than dropout-based ones.

Population-based training improves pragmatic language generation.

Method scales to large vocabularies and generalizes across new contexts.

Abstract

To be good conversational partners, natural language processing (NLP) systems should be trained to produce contextually useful utterances. Prior work has investigated training NLP systems with communication-based objectives, where a neural listener stands in as a communication partner. However, these systems commonly suffer from semantic drift where the learned language diverges radically from natural language. We propose a method that uses a population of neural listeners to regularize speaker training. We first show that language drift originates from the poor uncertainty calibration of a neural listener, which makes high-certainty predictions on novel sentences. We explore ensemble- and dropout-based populations of listeners and find that the former results in better uncertainty quantification. We evaluate both population-based objectives on reference games, and show that the ensemble method with better calibration enables the speaker to generate pragmatic utterances while scaling to a large vocabulary and generalizing to new games and listeners.