Modeling Multiple Support Strategies within a Single Turn for Emotional Support Conversations

TL;DR

This paper introduces methods for generating multi-strategy supportive utterances in emotional support conversations, demonstrating improved support quality through multi-strategy modeling and reinforcement learning.

Contribution

It formulates multi-strategy utterance generation for ESC, proposing All-in-One and One-by-One methods with reinforcement learning, and provides the first empirical evidence of their effectiveness.

Findings

Multi-strategy utterances improve supportive quality.

Proposed methods outperform single-strategy baselines.

Reinforcement learning enhances strategy selection and response quality.

Abstract

Emotional Support Conversation (ESC) aims to assist individuals experiencing distress by generating empathetic and supportive dialogue. While prior work typically assumes that each supporter turn corresponds to a single strategy, real-world supportive communication often involves multiple strategies within a single utterance. In this paper, we revisit the ESC task by formulating it as multi-strategy utterance generation, where each utterance may contain one or more strategy-response pairs. We propose two generation methods: All-in-One, which predicts all strategy-response pairs in a single decoding step, and One-by-One, which iteratively generates strategy-response pairs until completion. Both methods are further enhanced with cognitive reasoning guided by reinforcement learning to improve strategy selection and response composition. We evaluate our models on the ESConv dataset under both utterance-level and dialogue-level settings. Experimental results show that our methods effectively model multi-strategy utterances and lead to improved supportive quality and dialogue success. To our knowledge, this work provides the first systematic empirical evidence that allowing multiple support strategies within a single utterance is both feasible and beneficial for emotional support conversations. All code and data will be publicly available at https://github.com/aliyun/qwen-dianjin.