DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference Optimization

TL;DR

This paper introduces DecoupledESC, a framework that improves emotional support conversation generation by separating strategy planning and response generation, utilizing a new dataset and preference optimization to reduce errors and enhance response quality.

Contribution

It proposes a decoupled approach for ESC that leverages a new dataset and preference optimization, addressing data entanglement and ambiguity issues in existing methods.

Findings

Outperforms joint optimization baselines in experiments.

Reduces psychological errors in generated responses.

Enhances response quality through decoupled training.

Abstract

Recent advances in Emotional Support Conversation (ESC) have improved emotional support generation by fine-tuning Large Language Models (LLMs) via Supervised Fine-Tuning (SFT). However, common psychological errors still persist. While Direct Preference Optimization (DPO) shows promise in reducing such errors through pairwise preference learning, its effectiveness in ESC tasks is limited by two key challenges: (1) Entangled data structure: Existing ESC data inherently entangles psychological strategies and response content, making it difficult to construct high-quality preference pairs; and (2) Optimization ambiguity: Applying vanilla DPO to such entangled pairwise data leads to ambiguous training objectives. To address these issues, we introduce Inferential Preference Mining (IPM) to construct high-quality preference data, forming the IPM-PrefDial dataset. Building upon this data, we propose a Decoupled ESC framework inspired by Gross's Extended Process Model of Emotion Regulation, which decomposes the ESC task into two sequential subtasks: strategy planning and empathic response generation. Each was trained via SFT and subsequently enhanced by DPO to align with the psychological preference. Extensive experiments demonstrate that our Decoupled ESC framework outperforms joint optimization baselines, reducing preference bias and improving response quality.