A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots

TL;DR

This survey reviews the evolution of dialogue systems from simple rule-based models to complex deep learning architectures, highlighting challenges in evaluation practices and suggesting future research directions.

Contribution

It provides a comprehensive overview of dialogue system development, emphasizing the transition to neural models and discussing evaluation limitations and future research avenues.

Findings

Progressed from rule-based to deep learning systems

Evaluation metrics lack consistency and justification

Limited improvements observed on some metrics

Abstract

In spoken dialogue systems, we aim to deploy artificial intelligence to build automated dialogue agents that can converse with humans. Dialogue systems are increasingly being designed to move beyond just imitating conversation and also improve from such interactions over time. In this survey, we present a broad overview of methods developed to build dialogue systems over the years. Different use cases for dialogue systems ranging from task-based systems to open domain chatbots motivate and necessitate specific systems. Starting from simple rule-based systems, research has progressed towards increasingly complex architectures trained on a massive corpus of datasets, like deep learning systems. Motivated with the intuition of resembling human dialogues, progress has been made towards incorporating emotions into the natural language generator, using reinforcement learning. While we see a trend of highly marginal improvement on some metrics, we find that limited justification exists for the metrics, and evaluation practices are not uniform. To conclude, we flag these concerns and highlight possible research directions.