MeetSum: Transforming Meeting Transcript Summarization using Transformers!

TL;DR

This paper introduces MeetSum, a Transformer-based model with a Pointer Generator and coverage mechanism, that effectively summarizes meeting transcripts, outperforming previous models through zero-shot and fine-tuning strategies.

Contribution

The paper presents a novel Transformer-based summarization model for meetings that leverages zero-shot learning and out-of-domain pretraining to improve performance.

Findings

Achieved at least 5 ROUGE-2 score improvement over previous models.

Zero-shot training on news data outperforms training on meeting data.

Fine-tuning on meeting data further enhances summarization quality.

Abstract

Creating abstractive summaries from meeting transcripts has proven to be challenging due to the limited amount of labeled data available for training neural network models. Moreover, Transformer-based architectures have proven to beat state-of-the-art models in summarizing news data. In this paper, we utilize a Transformer-based Pointer Generator Network to generate abstract summaries for meeting transcripts. This model uses 2 LSTMs as an encoder and a decoder, a Pointer network which copies words from the inputted text, and a Generator network to produce out-of-vocabulary words (hence making the summary abstractive). Moreover, a coverage mechanism is used to avoid repetition of words in the generated summary. First, we show that training the model on a news summary dataset and using zero-shot learning to test it on the meeting dataset proves to produce better results than training it on the AMI meeting dataset. Second, we show that training this model first on out-of-domain data, such as the CNN-Dailymail dataset, followed by a fine-tuning stage on the AMI meeting dataset is able to improve the performance of the model significantly. We test our model on a testing set from the AMI dataset and report the ROUGE-2 score of the generated summary to compare with previous literature. We also report the Factual score of our summaries since it is a better benchmark for abstractive summaries since the ROUGE-2 score is limited to measuring word-overlaps. We show that our improved model is able to improve on previous models by at least 5 ROUGE-2 scores, which is a substantial improvement. Also, a qualitative analysis of the summaries generated by our model shows that these summaries and human-readable and indeed capture most of the important information from the transcripts.