Generative Adversarial Imitation Learning for Empathy-based AI

TL;DR

This paper introduces a novel GAIL-based model utilizing GPT-2 for empathetic, context-aware conversational AI that outperforms existing models in generating personalized empathetic responses.

Contribution

It applies generative adversarial imitation learning to fine-tune GPT-2 for empathy-driven dialogue generation, a novel approach in conversational AI.

Findings

Model achieves higher response scores than baselines.

Outperforms recent conversational AI models in multi-turn interactions.

Effective in generating concise, personalized empathetic responses.

Abstract

Generative adversarial imitation learning (GAIL) is a model-free algorithm that has been shown to provide strong results in imitating complex behaviors in high-dimensional environments. In this paper, we utilize the GAIL model for text generation to develop empathy-based context-aware conversational AI. Our model uses an expert trajectory of empathetic prompt-response dialogues which can accurately exhibit the correct empathetic emotion when generating a response. The Generator of the GAIL model uses the GPT-2 sequential pre-trained language model trained on 117 million parameters from 40 GB of internet data. We propose a novel application of an approach used in transfer learning to fine tune the GPT-2 model in order to generate concise, user-specific empathetic responses validated against the Discriminator. Our novel GAIL model utilizes a sentiment analysis history-based reinforcement learning approach to empathetically respond to human interactions in a personalized manner. We find that our model's response scores on various human-generated prompts collected from the Facebook Empathetic Dialogues dataset outperform baseline counterparts. Moreover, our model improves upon various history-based conversational AI models developed recently, as our model's performance over a sustained conversation of 3 or more interactions outperform similar conversational AI models.