Enhancing User Engagement in Socially-Driven Dialogue through Interactive LLM Alignments

TL;DR

This paper introduces a method to improve user engagement in socially-driven dialogues by aligning large language models with user reactions using a novel interaction simulation and preference optimization.

Contribution

It proposes a new approach that leverages future conversation signals and a user simulator with i×MCTS to directly optimize LLMs for user engagement.

Findings

Enhanced user engagement in emotional support and persuasion scenarios.

Effective alignment of LLMs using direct preference optimization.

Demonstrated improvements over baseline models.

Abstract

Enhancing user engagement through interactions plays an essential role in socially-driven dialogues. While prior works have optimized models to reason over relevant knowledge or plan a dialogue act flow, the relationship between user engagement and knowledge or dialogue acts is subtle and does not guarantee user engagement in socially-driven dialogues. To this end, we enable interactive LLMs to learn user engagement by leveraging signals from the future development of conversations. Specifically, we adopt a more direct and relevant indicator of user engagement, i.e., the user's reaction related to dialogue intention after the interaction, as a reward to align interactive LLMs. To achieve this, we develop a user simulator to interact with target interactive LLMs and explore interactions between the user and the interactive LLM system via \textit{i$\times$MCTS} (\textit{M}onte \textit{C}arlo \textit{T}ree \textit{S}earch for \textit{i}nteraction). In this way, we collect a dataset containing pairs of higher and lower-quality experiences using \textit{i$\times$MCTS}, and align interactive LLMs for high-level user engagement by direct preference optimization (DPO) accordingly. Experiments conducted on two socially-driven dialogue scenarios (emotional support conversations and persuasion for good) demonstrate that our method effectively enhances user engagement in interactive LLMs.