Controllable Generation of Dialogue Acts for Dialogue Systems via Few-Shot Response Generation and Ranking

TL;DR

This paper introduces a novel few-shot overgenerate-and-rank method for controllable dialogue act generation in dialogue systems, outperforming fine-tuning approaches in accuracy and semantic fidelity across multiple domains and models.

Contribution

The paper presents the first automatic ranking method for dialogue act and attribute accuracy in NLG, using prompt-based learning and pseudo-references for controllable response generation.

Findings

Several prompt styles achieve perfect DA accuracy

Semantic accuracy reaches 99.81% with the proposed method

Outperforms few-shot fine-tuning in experiments

Abstract

Dialogue systems need to produce responses that realize multiple types of dialogue acts (DAs) with high semantic fidelity. In the past, natural language generators (NLGs) for dialogue were trained on large parallel corpora that map from a domain-specific DA and its semantic attributes to an output utterance. Recent work shows that pretrained language models (LLMs) offer new possibilities for controllable NLG using prompt-based learning. Here we develop a novel few-shot overgenerate-and-rank approach that achieves the controlled generation of DAs. We compare eight few-shot prompt styles that include a novel method of generating from textual pseudo-references using a textual style transfer approach. We develop six automatic ranking functions that identify outputs with both the correct DA and high semantic accuracy at generation time. We test our approach on three domains and four LLMs. To our knowledge, this is the first work on NLG for dialogue that automatically ranks outputs using both DA and attribute accuracy. For completeness, we compare our results to fine-tuned few-shot models trained with 5 to 100 instances per DA. Our results show that several prompt settings achieve perfect DA accuracy, and near perfect semantic accuracy (99.81%) and perform better than few-shot fine-tuning.