PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanism

TL;DR

This paper introduces PVGRU, a novel recurrent unit that captures subtle semantic variability in dialogue responses, leading to more diverse and relevant chatbot replies.

Contribution

It proposes a pseudo-variational mechanism within GRUs to better model variability, enhancing response diversity and relevance in dialogue systems.

Findings

PVGRU improves response diversity and relevance.

The PVHD model outperforms baselines on benchmark datasets.

The approach effectively captures subtle semantic differences.

Abstract

We investigate response generation for multi-turn dialogue in generative-based chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the sequences, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) component without posterior knowledge through introducing a recurrent summarizing variable into the GRU, which can aggregate the accumulated distribution variations of subsequences. PVGRU can perceive the subtle semantic variability through summarizing variables that are optimized by the devised distribution consistency and reconstruction objectives. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.