Persona-Aware Alignment Framework for Personalized Dialogue Generation

TL;DR

This paper introduces PAL, a novel framework for personalized dialogue generation that explicitly aligns responses with personas, significantly improving persona relevance and consistency over existing models.

Contribution

The paper proposes a new training framework that directly optimizes persona alignment, enhancing personalization in dialogue generation beyond implicit token-level methods.

Findings

Outperforms state-of-the-art personalized dialogue models

Enhances persona relevance and consistency in generated responses

Effective two-stage training and inference strategy

Abstract

Personalized dialogue generation aims to leverage persona profiles and dialogue history to generate persona-relevant and consistent responses. Mainstream models typically rely on token-level language model training with persona dialogue data, such as Next Token Prediction, to implicitly achieve personalization, making these methods tend to neglect the given personas and generate generic responses. To address this issue, we propose a novel Persona-Aware Alignment Framework (PAL), which directly treats persona alignment as the training objective of dialogue generation. Specifically, PAL employs a two-stage training method including Persona-aware Learning and Persona Alignment, equipped with an easy-to-use inference strategy Select then Generate, to improve persona sensitivity and generate more persona-relevant responses at the semantics level. Through extensive experiments, we demonstrate that our framework outperforms many state-of-the-art personalized dialogue methods and large language models.