Generate, Evaluate, and Select: A Dialogue System with a Response Evaluator for Diversity-Aware Response Generation

TL;DR

This paper introduces a dialogue system that generates multiple diverse responses, evaluates them with a response evaluator, and selects the best, leading to more engaging and varied conversations.

Contribution

It presents a novel generator-evaluator framework that improves response diversity and quality in dialogue systems, validated through human evaluations.

Findings

Responses were often judged better than baseline

The system increased response diversity

Human evaluations confirmed effectiveness

Abstract

We aim to overcome the lack of diversity in responses of current dialogue systems and to develop a dialogue system that is engaging as a conversational partner. We propose a generator-evaluator model that evaluates multiple responses generated by a response generator and selects the best response by an evaluator. By generating multiple responses, we obtain diverse responses. We conduct human evaluations to compare the output of the proposed system with that of a baseline system. The results of the human evaluations showed that the proposed system's responses were often judged to be better than the baseline system's, and indicated the effectiveness of the proposed method.