CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision

TL;DR

CiteSum introduces an automatic method to generate ultra-short scientific summaries from citation texts, creating a large benchmark dataset and demonstrating effective domain adaptation with limited supervision, outperforming existing methods.

Contribution

The paper presents a novel automatic extraction approach for scientific TLDRs, creating the large CiteSum dataset and showing strong zero-shot and few-shot summarization performance.

Findings

CiteSum is around 30 times larger than previous datasets.

CITES outperforms fully-supervised methods on SciTLDR without fine-tuning.

CITES achieves state-of-the-art results with only 128 examples.

Abstract

Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific papers. Previous efforts on curating scientific TLDR datasets failed to scale up due to the heavy human annotation and domain expertise required. In this paper, we propose a simple yet effective approach to automatically extracting TLDR summaries for scientific papers from their citation texts. Based on the proposed approach, we create a new benchmark CiteSum without human annotation, which is around 30 times larger than the previous human-curated dataset SciTLDR. We conduct a comprehensive analysis of CiteSum, examining its data characteristics and establishing strong baselines. We further demonstrate the usefulness of CiteSum by adapting models pre-trained on CiteSum (named CITES) to new tasks and domains with limited supervision. For scientific extreme summarization, CITES outperforms most fully-supervised methods on SciTLDR without any fine-tuning and obtains state-of-the-art results with only 128 examples. For news extreme summarization, CITES achieves significant gains on XSum over its base model (not pre-trained on CiteSum), e.g., +7.2 ROUGE-1 zero-shot performance and state-of-the-art few-shot performance. For news headline generation, CITES performs the best among unsupervised and zero-shot methods on Gigaword. Our dataset and code can be found at https://github.com/morningmoni/CiteSum.