SentiCite: An Approach for Publication Sentiment Analysis

TL;DR

This paper introduces SentiCite, a novel system for analyzing the sentiment and intent behind citations in scientific papers, addressing the qualitative aspect often overlooked by quantitative scientometric measures.

Contribution

It presents a new sentiment analysis system for citations, along with two annotated datasets, and demonstrates improved performance over existing methods.

Findings

SentiCite achieves an F1-measure of 0.71 in sentiment analysis.

The system effectively detects citation intent such as dataset references.

Datasets contain about 2,600 citations with ground truth annotations.

Abstract

With the rapid growth in the number of scientific publications, year after year, it is becoming increasingly difficult to identify quality authoritative work on a single topic. Though there is an availability of scientometric measures which promise to offer a solution to this problem, these measures are mostly quantitative and rely, for instance, only on the number of times an article is cited. With this approach, it becomes irrelevant if an article is cited 10 times in a positive, negative or neutral way. In this context, it is quite important to study the qualitative aspect of a citation to understand its significance. This paper presents a novel system for sentiment analysis of citations in scientific documents (SentiCite) and is also capable of detecting nature of citations by targeting the motivation behind a citation, e.g., reference to a dataset, reading reference. Furthermore, the paper also presents two datasets (SentiCiteDB and IntentCiteDB) containing about 2,600 citations with their ground truth for sentiment and nature of citation. SentiCite along with other state-of-the-art methods for sentiment analysis are evaluated on the presented datasets. Evaluation results reveal that SentiCite outperforms state-of-the-art methods for sentiment analysis in scientific publications by achieving a F1-measure of 0.71.