From Generic Empathy to Personalized Emotional Support: A Self-Evolution Framework for User Preference Alignment

TL;DR

This paper introduces a self-evolution framework for large language models to improve personalized emotional support by aligning responses with individual user preferences through iterative self-refinement.

Contribution

It presents a novel two-phase self-evolution approach enabling LLMs to better understand and adapt to user preferences in emotional support scenarios.

Findings

Significant improvement in emotional support quality

Reduction in unhelpful and generic responses

Enhanced alignment with user preferences

Abstract

Effective emotional support hinges on understanding users' emotions and needs to provide meaningful comfort during multi-turn interactions. Large Language Models (LLMs) show great potential for expressing empathy; however, they often deliver generic and one-size-fits-all responses that fail to address users' specific needs. To tackle this issue, we propose a self-evolution framework designed to help LLMs improve their responses to better align with users' implicit preferences concerning user profiles (personalities), emotional states, and specific situations. Our framework consists of two distinct phases: \textit{(1)} \textit{Emotional Support Experience Acquisition}, where LLMs are fine-tuned on limited emotional support conversation data to provide basic support, and \textit{(2)} \textit{Self-Improvement for Personalized Emotional Support}, where LLMs leverage self-reflection and self-refinement to generate personalized responses. Through iterative direct preference optimization between the pre- and post-refined responses, our model generates responses that reflect a better understanding of the user's implicit preferences. Extensive experiments and evaluations demonstrate that our method significantly enhances the model's performance in emotional support, reducing unhelpful responses and minimizing discrepancies between user preferences and model outputs.