Improving Generalization in Task-oriented Dialogues with Workflows and Action Plans

TL;DR

This paper enhances task-oriented dialogue models by incorporating workflows and action plans, significantly improving their ability to generalize to unseen multi-step tasks and actions, which traditional end-to-end models struggle with.

Contribution

The authors introduce a method of augmenting dialogue context with workflows and action plans, enabling better generalization to new tasks and actions in multi-step dialogues.

Findings

Models with action plans outperform baseline models on unseen workflows.

Inclusion of action plans improves execution of unseen actions.

Augmentation with workflows enhances multi-step task completion.

Abstract

Task-oriented dialogue is difficult in part because it involves understanding user intent, collecting information from the user, executing API calls, and generating helpful and fluent responses. However, for complex tasks one must also correctly do all of these things over multiple steps, and in a specific order. While large pre-trained language models can be fine-tuned end-to-end to create multi-step task-oriented dialogue agents that generate fluent text, our experiments confirm that this approach alone cannot reliably perform new multi-step tasks that are unseen during training. To address these limitations, we augment the dialogue contexts given to \textmd{text2text} transformers with known \textit{valid workflow names} and \textit{action plans}. Action plans consist of sequences of actions required to accomplish a task, and are encoded as simple sequences of keywords (e.g. verify-identity, pull-up-account, reset-password, etc.). We perform extensive experiments on the Action-Based Conversations Dataset (ABCD) with T5-small, base and large models, and show that such models: a) are able to more readily generalize to unseen workflows by following the provided plan, and b) are able to generalize to executing unseen actions if they are provided in the plan. In contrast, models are unable to fully accomplish new multi-step tasks when they are not provided action plan information, even when given new valid workflow names.