Empathetic Dialogue Generation with Pre-trained RoBERTa-GPT2 and External Knowledge

TL;DR

This paper introduces a novel empathetic dialogue generation model using pre-trained RoBERTa as encoder and GPT-2 as decoder, enhanced with external commonsense and emotional knowledge, achieving state-of-the-art emotion accuracy.

Contribution

It combines pre-trained encoder-decoder architecture with external knowledge extraction to improve empathetic response generation in dialogue systems.

Findings

Achieves new state-of-the-art emotion accuracy.

External knowledge improves empathetic response quality.

Pre-trained models enhance dialogue understanding and generation.

Abstract

One challenge for dialogue agents is to recognize feelings of the conversation partner and respond accordingly. In this work, RoBERTa-GPT2 is proposed for empathetic dialogue generation, where the pre-trained auto-encoding RoBERTa is utilised as encoder and the pre-trained auto-regressive GPT-2 as decoder. With the combination of the pre-trained RoBERTa and GPT-2, our model realizes a new state-of-the-art emotion accuracy. To enable the empathetic ability of RoBERTa-GPT2 model, we propose a commonsense knowledge and emotional concepts extractor, in which the commonsensible and emotional concepts of dialogue context are extracted for the GPT-2 decoder. The experiment results demonstrate that the empathetic dialogue generation benefits from both pre-trained encoder-decoder architecture and external knowledge.