Are You Listening to Me? Fine-Tuning Chatbots for Empathetic Dialogue

TL;DR

This paper investigates how large language models can be fine-tuned to generate empathetic dialogue, emphasizing the importance of combining automated analysis with human judgment to improve emotional intelligence in conversational agents.

Contribution

It introduces a methodology for extending empathic dialogue datasets using LLMs and analyzes emotional progression through sentiment analysis and expert evaluation.

Findings

Generated dialogues often reflected the intended emotional structure

Human evaluation revealed differences in perceived empathy and coherence

Emotion modeling requires both structural and qualitative depth

Abstract

Conversational agents have made significant progress since ELIZA, expanding their role across various domains, including healthcare, education, and customer service. As these agents become increasingly integrated into daily human interactions, the need for emotional intelligence, particularly empathetic listening, becomes increasingly essential. In this study, we explore how Large Language Models (LLMs) respond when tasked with generating emotionally rich interactions. Starting from a small dataset manually crafted by an expert to reflect empathic behavior, we extended the conversations using two LLMs: ChatGPT and Gemini. We analyzed the emotional progression of the dialogues using both sentiment analysis (via VADER) and expert assessments. While the generated conversations often mirrored the intended emotional structure, human evaluation revealed important differences in the perceived empathy and coherence of the responses. These findings suggest that emotion modeling in dialogues requires not only structural alignment in the expressed emotions but also qualitative depth, highlighting the importance of combining automated and humancentered methods in the development of emotionally competent agents.