A Neural Conversational Model

TL;DR

This paper introduces a sequence-to-sequence neural model for conversational AI that can learn from large datasets, generating simple, domain-specific, and open-domain conversations with minimal handcrafted rules.

Contribution

It presents a straightforward end-to-end neural approach for conversational modeling that works across domains and requires less manual rule crafting compared to prior methods.

Findings

The model can generate simple conversations from large datasets.

It performs well on domain-specific and open-domain datasets.

Lacks consistency, which is a common failure mode.

Abstract

Conversational modeling is an important task in natural language understanding and machine intelligence. Although previous approaches exist, they are often restricted to specific domains (e.g., booking an airline ticket) and require hand-crafted rules. In this paper, we present a simple approach for this task which uses the recently proposed sequence to sequence framework. Our model converses by predicting the next sentence given the previous sentence or sentences in a conversation. The strength of our model is that it can be trained end-to-end and thus requires much fewer hand-crafted rules. We find that this straightforward model can generate simple conversations given a large conversational training dataset. Our preliminary results suggest that, despite optimizing the wrong objective function, the model is able to converse well. It is able extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles. On a domain-specific IT helpdesk dataset, the model can find a solution to a technical problem via conversations. On a noisy open-domain movie transcript dataset, the model can perform simple forms of common sense reasoning. As expected, we also find that the lack of consistency is a common failure mode of our model.