Incidental or influential? - Challenges in automatically detecting citation importance using publication full texts

TL;DR

This paper critically evaluates various features used in automating citation importance detection from full texts, highlighting the predictive power of in-text references and abstract similarity, and discusses challenges and future directions.

Contribution

It provides a comprehensive analysis of existing features for citation importance classification and emphasizes the need for a large-scale gold standard dataset.

Findings

Number of in-text references is highly predictive of citation influence

Abstract similarity is a strong predictor, contrary to previous studies

Many features previously used are not particularly predictive

Abstract

This work looks in depth at several studies that have attempted to automate the process of citation importance classification based on the publications full text. We analyse a range of features that have been previously used in this task. Our experimental results confirm that the number of in text references are highly predictive of influence. Contrary to the work of Valenzuela et al. we find abstract similarity one of the most predictive features. Overall, we show that many of the features previously described in literature are not particularly predictive. Consequently, we discuss challenges and potential improvements in the classification pipeline, provide a critical review of the performance of individual features and address the importance of constructing a large scale gold standard reference dataset.