Self-Attention-Based Message-Relevant Response Generation for Neural Conversation Model

TL;DR

This paper introduces a self-attention-based method to enhance message relevance and diversity in neural conversation models, addressing the issue of generic responses and demonstrating improved performance across multiple metrics.

Contribution

It proposes a novel self-attention mechanism to promote message relevance and diversity in neural dialogue generation, with an in-depth analysis of its effectiveness.

Findings

Improved response relevance and diversity in neural conversation models.

Self-attention mechanism outperforms standard methods in evaluation metrics.

Efficient and effective approach for dialogue generation enhancement.

Abstract

Using a sequence-to-sequence framework, many neural conversation models for chit-chat succeed in naturalness of the response. Nevertheless, the neural conversation models tend to give generic responses which are not specific to given messages, and it still remains as a challenge. To alleviate the tendency, we propose a method to promote message-relevant and diverse responses for neural conversation model by using self-attention, which is time-efficient as well as effective. Furthermore, we present an investigation of why and how effective self-attention is in deep comparison with the standard dialogue generation. The experiment results show that the proposed method improves the standard dialogue generation in various evaluation metrics.