Dialog Acts for Task-Driven Embodied Agents

TL;DR

This paper introduces a new set of dialog acts for embodied agents, annotates a large dataset with these acts, and demonstrates their effectiveness in improving task success rates in embodied task completion.

Contribution

It presents TEACh-DA, a large-scale dataset with dialog act annotations for embodied task-oriented conversations, and shows how these annotations enhance task performance.

Findings

Dialog acts improve task success rate by up to 2 points.

TEACh-DA is one of the first large-scale dialog act datasets for embodied tasks.

Models trained with dialog acts better predict actions and responses.

Abstract

Embodied agents need to be able to interact in natural language understanding task descriptions and asking appropriate follow up questions to obtain necessary information to be effective at successfully accomplishing tasks for a wide range of users. In this work, we propose a set of dialog acts for modelling such dialogs and annotate the TEACh dataset that includes over 3,000 situated, task oriented conversations (consisting of 39.5k utterances in total) with dialog acts. TEACh-DA is one of the first large scale dataset of dialog act annotations for embodied task completion. Furthermore, we demonstrate the use of this annotated dataset in training models for tagging the dialog acts of a given utterance, predicting the dialog act of the next response given a dialog history, and use the dialog acts to guide agent's non-dialog behaviour. In particular, our experiments on the TEACh Execution from Dialog History task where the model predicts the sequence of low level actions to be executed in the environment for embodied task completion, demonstrate that dialog acts can improve end task success rate by up to 2 points compared to the system without dialog acts.