End to End Dialogue Transformer

TL;DR

This paper introduces an end-to-end dialogue system leveraging Transformer architecture, replacing RNN-based models like Sequicity, to improve conversational AI capabilities.

Contribution

It presents a novel Transformer-based dialogue system that operates in an end-to-end sequence-to-sequence manner, enhancing upon prior RNN-based approaches.

Findings

Transformer-based model achieves competitive dialogue generation.

End-to-end architecture simplifies the dialogue pipeline.

Potential improvements in response quality and coherence.

Abstract

Dialogue systems attempt to facilitate conversations between humans and computers, for purposes as diverse as small talk to booking a vacation. We are here inspired by the performance of the recurrent neural network-based model Sequicity, which when conducting a dialogue uses a sequence-to-sequence architecture to first produce a textual representation of what is going on in the dialogue, and in a further step use this along with database findings to produce a reply to the user. We here propose a dialogue system based on the Transformer architecture instead of Sequicity's RNN-based architecture, that works similarly in an end-to-end, sequence-to-sequence fashion.