Summaformers @ LaySumm 20, LongSumm 20

TL;DR

This paper introduces Transformer-based systems for summarizing scientific research papers into layman and detailed summaries, demonstrating effectiveness through ROUGE metrics and top rankings in blind tests.

Contribution

It presents a simple, section-based Transformer approach tailored for two types of scientific paper summaries, outperforming existing methods in evaluations.

Findings

System ranks first for LongSumm and third for LaySumm in blind tests.

ROUGE metrics confirm the effectiveness of the proposed summarization approach.

Section contribution modeling improves summary quality.

Abstract

Automatic text summarization has been widely studied as an important task in natural language processing. Traditionally, various feature engineering and machine learning based systems have been proposed for extractive as well as abstractive text summarization. Recently, deep learning based, specifically Transformer-based systems have been immensely popular. Summarization is a cognitively challenging task - extracting summary worthy sentences is laborious, and expressing semantics in brief when doing abstractive summarization is complicated. In this paper, we specifically look at the problem of summarizing scientific research papers from multiple domains. We differentiate between two types of summaries, namely, (a) LaySumm: A very short summary that captures the essence of the research paper in layman terms restricting overtly specific technical jargon and (b) LongSumm: A much longer detailed summary aimed at providing specific insights into various ideas touched upon in the paper. While leveraging latest Transformer-based models, our systems are simple, intuitive and based on how specific paper sections contribute to human summaries of the two types described above. Evaluations against gold standard summaries using ROUGE metrics prove the effectiveness of our approach. On blind test corpora, our system ranks first and third for the LongSumm and LaySumm tasks respectively.