Why Did You Not Compare With That? Identifying Papers for Use as Baselines

TL;DR

This paper introduces a neural classification approach to automatically identify baseline papers in scientific articles, addressing the challenge of diverse citation appearances and outperforming existing methods.

Contribution

It presents a new dataset of annotated references and a multi-module attention neural classifier for baseline identification, advancing citation role classification.

Findings

The classifier outperforms four state-of-the-art methods.

A new dataset of 2,075 papers with annotated references is created.

Analysis reveals key challenges in baseline identification.

Abstract

We propose the task of automatically identifying papers used as baselines in a scientific article. We frame the problem as a binary classification task where all the references in a paper are to be classified as either baselines or non-baselines. This is a challenging problem due to the numerous ways in which a baseline reference can appear in a paper. We develop a dataset of $2,075$ papers from ACL anthology corpus with all their references manually annotated as one of the two classes. We develop a multi-module attention-based neural classifier for the baseline classification task that outperforms four state-of-the-art citation role classification methods when applied to the baseline classification task. We also present an analysis of the errors made by the proposed classifier, eliciting the challenges that make baseline identification a challenging problem.