Attention with Intention for a Neural Network Conversation Model

TL;DR

This paper introduces a neural network conversation model that integrates attention and intention mechanisms through three recurrent networks, enabling more natural and context-aware responses without labeled data.

Contribution

It presents a novel neural architecture combining attention and intention modeling with end-to-end training for dialogue generation.

Findings

Generates more natural responses in conversation tasks.

Effectively models intention dynamics during dialogue.

Achieves improved response relevance without labeled data.

Abstract

In a conversation or a dialogue process, attention and intention play intrinsic roles. This paper proposes a neural network based approach that models the attention and intention processes. It essentially consists of three recurrent networks. The encoder network is a word-level model representing source side sentences. The intention network is a recurrent network that models the dynamics of the intention process. The decoder network is a recurrent network produces responses to the input from the source side. It is a language model that is dependent on the intention and has an attention mechanism to attend to particular source side words, when predicting a symbol in the response. The model is trained end-to-end without labeling data. Experiments show that this model generates natural responses to user inputs.