Selective Prompting Tuning for Personalized Conversations with LLMs

TL;DR

This paper introduces Selective Prompt Tuning (SPT), a novel method that adaptively selects soft prompts for personalized conversations with LLMs, significantly improving response diversity and coherence.

Contribution

The paper proposes SPT, a dynamic soft prompt selection framework with contrastive learning, to enhance personalization and diversity in LLM-based dialogues, addressing limitations of prompting and fine-tuning.

Findings

Response diversity increased by up to 90%

SPT outperforms baseline methods in key metrics

Code is publicly available for further research

Abstract

In conversational AI, personalizing dialogues with persona profiles and contextual understanding is essential. Despite large language models' (LLMs) improved response coherence, effective persona integration remains a challenge. In this work, we first study two common approaches for personalizing LLMs: textual prompting and direct fine-tuning. We observed that textual prompting often struggles to yield responses that are similar to the ground truths in datasets, while direct fine-tuning tends to produce repetitive or overly generic replies. To alleviate those issues, we propose \textbf{S}elective \textbf{P}rompt \textbf{T}uning (SPT), which softly prompts LLMs for personalized conversations in a selective way. Concretely, SPT initializes a set of soft prompts and uses a trainable dense retriever to adaptively select suitable soft prompts for LLMs according to different input contexts, where the prompt retriever is dynamically updated through feedback from the LLMs. Additionally, we propose context-prompt contrastive learning and prompt fusion learning to encourage the SPT to enhance the diversity of personalized conversations. Experiments on the CONVAI2 dataset demonstrate that SPT significantly enhances response diversity by up to 90\%, along with improvements in other critical performance indicators. Those results highlight the efficacy of SPT in fostering engaging and personalized dialogue generation. The SPT model code (https://github.com/hqsiswiliam/SPT) is publicly available for further exploration.