Attribute Controlled Dialogue Prompting

TL;DR

This paper introduces an instance-specific prompt-tuning method for dialogue generation that adapts prompts based on control codes, improving performance while using significantly fewer parameters than traditional fine-tuning.

Contribution

It proposes a novel prompt-tuning algorithm that generates instance-specific prompts for dialogue, addressing variability in input data and enhancing control over generated responses.

Findings

Outperforms baseline prompt methods in automated and human evaluations.

Achieves comparable results to fine-tuning with only 5-6% of parameters.

Effective in open-domain dialogue datasets.

Abstract

Prompt-tuning has become an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks. However, both discrete prompting and continuous prompting assume fixed prompts for all data samples within a task, neglecting the fact that inputs vary greatly in some tasks such as open-domain dialogue generation. In this paper, we present a novel, instance-specific prompt-tuning algorithm for dialogue generation. Specifically, we generate prompts based on instance-level control code, rather than the conversation history, to explore their impact on controlled dialogue generation. Experiments on popular open-domain dialogue datasets, evaluated on both automated metrics and human evaluation, demonstrate that our method is superior to prompting baselines and comparable to fine-tuning with only 5%-6% of total parameters.