Steering Conversational Large Language Models for Long Emotional Support Conversations

TL;DR

This paper investigates how large language models can be guided to follow emotional support strategies consistently in long conversations, introducing new metrics and datasets to improve steerability.

Contribution

We propose the Strategy Relevant Attention (SRA) metric, create a strategy-conditioned dataset, and develop a fine-tuned model that improves adherence to emotional support strategies.

Findings

The SRA metric effectively measures strategy adherence.

Fine-tuning improves model steerability in extended conversations.

Publicly available code and data facilitate further research.

Abstract

In this study, we address the challenge of enabling large language models (LLMs) to consistently adhere to emotional support strategies in extended conversations. We focus on the steerability of the Llama-2 and Llama-3 suite of models, examining their ability to maintain these strategies throughout interactions. To assess this, we introduce the Strategy Relevant Attention (SRA) metric, which quantifies the model's adherence to the prompted strategy through attention maps. To facilitate our study, we create a strategy-conditioned synthetic conversational dataset derived from the ESConv dataset. We also propose various baselines informed by our proposed SRA metric to address the challenge and propose a fine-tuned model that significantly enhances the steerability of the base model in following the strategy throughout the conversation. The code and data are publicly available on our GitHub.