Neural Discourse Modeling of Conversations

TL;DR

This paper enhances neural conversation models by extending RNNs to capture long-range discourse, demonstrating improved coherence and cohesion through added context and layers.

Contribution

It introduces an extended RNN-based sequence model for discourse modeling and provides analysis on the impact of context and model depth on conversation quality.

Findings

Adding an RNN layer improves output quality

More context leads to better performance

Models show increased discourse coherence

Abstract

Deep neural networks have shown recent promise in many language-related tasks such as the modeling of conversations. We extend RNN-based sequence to sequence models to capture the long range discourse across many turns of conversation. We perform a sensitivity analysis on how much additional context affects performance, and provide quantitative and qualitative evidence that these models are able to capture discourse relationships across multiple utterances. Our results quantifies how adding an additional RNN layer for modeling discourse improves the quality of output utterances and providing more of the previous conversation as input also improves performance. By searching the generated outputs for specific discourse markers we show how neural discourse models can exhibit increased coherence and cohesion in conversations.