The CL-SciSumm Shared Task 2018: Results and Key Insights

TL;DR

This paper reports the results of the 2018 CL-SciSumm shared task, a scientific document summarization challenge in computational linguistics, analyzing system performances on a dataset of 60 annotated research papers.

Contribution

It presents the first medium-scale shared task on CL domain scientific summarization, providing a benchmark dataset and evaluation framework for future research.

Findings

Multiple systems evaluated with two metrics

Insights into system strengths and weaknesses

Benchmark results for CL scientific summarization

Abstract

This overview describes the official results of the CL-SciSumm Shared Task 2018 -- the first medium-scale shared task on scientific document summarization in the computational linguistics (CL) domain. This year, the dataset comprised 60 annotated sets of citing and reference papers from the open access research papers in the CL domain. The Shared Task was organized as a part of the 41st Annual Conference of the Special Interest Group in Information Retrieval (SIGIR), held in Ann Arbor, USA in July 2018. We compare the participating systems in terms of two evaluation metrics. The annotated dataset and evaluation scripts can be accessed and used by the community from: \url{https://github.com/WING-NUS/scisumm-corpus}.