Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints

TL;DR

This paper introduces a method to enhance neural conversation models by incorporating distributional constraints, leading to more content-rich and less generic responses without losing plausibility, validated through automatic and human evaluations.

Contribution

It proposes a novel approach using distributional constraints based on syntax, topics, and semantic similarity to improve response diversity in neural conversation models.

Findings

Responses are less generic and more content-rich.

The approach outperforms baseline models on automatic metrics.

Human judgments favor the diversity and relevance of generated responses.

Abstract

Neural conversation models tend to generate safe, generic responses for most inputs. This is due to the limitations of likelihood-based decoding objectives in generation tasks with diverse outputs, such as conversation. To address this challenge, we propose a simple yet effective approach for incorporating side information in the form of distributional constraints over the generated responses. We propose two constraints that help generate more content rich responses that are based on a model of syntax and topics (Griffiths et al., 2005) and semantic similarity (Arora et al., 2016). We evaluate our approach against a variety of competitive baselines, using both automatic metrics and human judgments, showing that our proposed approach generates responses that are much less generic without sacrificing plausibility. A working demo of our code can be found at https://github.com/abaheti95/DC-NeuralConversation.