Another Diversity-Promoting Objective Function for Neural Dialogue Generation

TL;DR

This paper introduces the Inverse Token Frequency loss, a new objective function that promotes diversity in neural dialogue generation by encouraging the use of rare tokens, achieving state-of-the-art diversity scores.

Contribution

The paper proposes the ITF loss function that scales loss based on token frequency, improving diversity without complicating the model or training process.

Findings

Achieves state-of-the-art DIST-1 score of 7.56 on OpenSubtitles dataset.

Maintains good BLEU-1 score while increasing diversity.

Achieves comparable DIST-1 scores to ground truth on Japanese Twitter replies dataset.

Abstract

Although generation-based dialogue systems have been widely researched, the response generations by most existing systems have very low diversities. The most likely reason for this problem is Maximum Likelihood Estimation (MLE) with Softmax Cross-Entropy (SCE) loss. MLE trains models to generate the most frequent responses from enormous generation candidates, although in actual dialogues there are various responses based on the context. In this paper, we propose a new objective function called Inverse Token Frequency (ITF) loss, which individually scales smaller loss for frequent token classes and larger loss for rare token classes. This function encourages the model to generate rare tokens rather than frequent tokens. It does not complicate the model and its training is stable because we only replace the objective function. On the OpenSubtitles dialogue dataset, our loss model establishes a state-of-the-art DIST-1 of 7.56, which is the unigram diversity score, while maintaining a good BLEU-1 score. On a Japanese Twitter replies dataset, our loss model achieves a DIST-1 score comparable to the ground truth.