Improving Computer Generated Dialog with Auxiliary Loss Functions and Custom Evaluation Metrics

TL;DR

This paper presents an improved neural dialog generation system that uses auxiliary loss functions and custom evaluation metrics to enhance coherence and cohesion in chatbot responses.

Contribution

It introduces novel auxiliary loss functions based on MMI and entropy, along with custom evaluation metrics for better dialog quality assessment.

Findings

Enhanced dialog coherence and cohesion demonstrated

Auxiliary loss functions improve response quality

Custom metrics correlate with human judgments

Abstract

Although people have the ability to engage in vapid dialogue without effort, this may not be a uniquely human trait. Since the 1960's researchers have been trying to create agents that can generate artificial conversation. These programs are commonly known as chatbots. With increasing use of neural networks for dialog generation, some conclude that this goal has been achieved. This research joins the quest by creating a dialog generating Recurrent Neural Network (RNN) and by enhancing the ability of this network with auxiliary loss functions and a beam search. Our custom loss functions achieve better cohesion and coherence by including calculations of Maximum Mutual Information (MMI) and entropy. We demonstrate the effectiveness of this system by using a set of custom evaluation metrics inspired by an abundance of previous research and based on tried-and-true principles of Natural Language Processing.