Abstractive and Extractive Text Summarization using Document Context Vector and Recurrent Neural Networks

TL;DR

This paper introduces a novel document-context based Seq2Seq model using RNNs for abstractive and extractive summarization, leveraging contextual information to produce more document-centric summaries and employing semi-supervised training for scalability.

Contribution

The study proposes a new document-contextual Seq2Seq approach that improves summarization quality and introduces a semi-supervised training method to scale the generation of training data.

Findings

Contextual models outperform standard Seq2Seq models.

Semi-supervised summaries are comparable to human summaries.

Techniques are effective for large document summarization.

Abstract

Sequence to sequence (Seq2Seq) learning has recently been used for abstractive and extractive summarization. In current study, Seq2Seq models have been used for eBay product description summarization. We propose a novel Document-Context based Seq2Seq models using RNNs for abstractive and extractive summarizations. Intuitively, this is similar to humans reading the title, abstract or any other contextual information before reading the document. This gives humans a high-level idea of what the document is about. We use this idea and propose that Seq2Seq models should be started with contextual information at the first time-step of the input to obtain better summaries. In this manner, the output summaries are more document centric, than being generic, overcoming one of the major hurdles of using generative models. We generate document-context from user-behavior and seller provided information. We train and evaluate our models on human-extracted-golden-summaries. The document-contextual Seq2Seq models outperform standard Seq2Seq models. Moreover, generating human extracted summaries is prohibitively expensive to scale, we therefore propose a semi-supervised technique for extracting approximate summaries and using it for training Seq2Seq models at scale. Semi-supervised models are evaluated against human extracted summaries and are found to be of similar efficacy. We provide side by side comparison for abstractive and extractive summarizers (contextual and non-contextual) on same evaluation dataset. Overall, we provide methodologies to use and evaluate the proposed techniques for large document summarization. Furthermore, we found these techniques to be highly effective, which is not the case with existing techniques.