Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems

TL;DR

This paper introduces SUIT, an iterative training method for task-oriented dialog systems that leverages subgoal identification and high-quality data sampling to enhance performance, achieving state-of-the-art results.

Contribution

The paper presents a novel iterative training approach that uses subgoal-aware sampling and distant supervision to improve dialog system training beyond static datasets.

Findings

SUIT achieves state-of-the-art performance on a ToD benchmark.

Iterative data generation enhances dialog system training.

Subgoal-aware sampling improves training data quality.

Abstract

Task-oriented Dialog (ToD) systems have to solve multiple subgoals to accomplish user goals, whereas feedback is often obtained only at the end of the dialog. In this work, we propose SUIT (SUbgoal-aware ITerative Training), an iterative training approach for improving ToD systems. We sample dialogs from the model we aim to improve and determine subgoals that contribute to dialog success using distant supervision to obtain high quality training samples. We show how this data improves supervised fine-tuning or, alternatively, preference learning results. SUIT is able to iteratively generate more data instead of relying on fixed static sets. SUIT reaches new state-of-the-art performance on a popular ToD benchmark.