Towards Automated Customer Support

TL;DR

This paper compares retrieval, seq2seq, and Transformer models for automated customer support, demonstrating that seq2seq models outperform others in semantic relevance on a large Twitter dataset.

Contribution

It provides an empirical comparison of three models for customer support chatbots, highlighting the effectiveness of seq2seq models over retrieval and Transformer approaches.

Findings

Seq2seq models outperform retrieval and Transformer models in semantic relevance.

Seq2seq models achieve higher word overlap with customer queries.

The study uses a large dataset of over two million Twitter support posts.

Abstract

Recent years have seen growing interest in conversational agents, such as chatbots, which are a very good fit for automated customer support because the domain in which they need to operate is narrow. This interest was in part inspired by recent advances in neural machine translation, esp. the rise of sequence-to-sequence (seq2seq) and attention-based models such as the Transformer, which have been applied to various other tasks and have opened new research directions in question answering, chatbots, and conversational systems. Still, in many cases, it might be feasible and even preferable to use simple information retrieval techniques. Thus, here we compare three different models:(i) a retrieval model, (ii) a sequence-to-sequence model with attention, and (iii) Transformer. Our experiments with the Twitter Customer Support Dataset, which contains over two million posts from customer support services of twenty major brands, show that the seq2seq model outperforms the other two in terms of semantics and word overlap.