The Role of Positive and Negative Citations in Scientific Evaluation

TL;DR

This paper introduces PANDORA, a novel algorithm that distinguishes positive and negative citation relationships, improving the accuracy of scientific impact assessment by accounting for potential manipulation through negative citations.

Contribution

The study develops PANDORA, which incorporates COI relationships and citation dynamics into impact evaluation, enhancing fairness and objectivity over existing methods.

Findings

PANDORA outperforms current solutions in Recommendation Intensity at top-K.

It achieves higher Spearman's rank correlation coefficient at top-K.

The method effectively weakens the influence of negative COI relationships.

Abstract

Quantifying the impact of scientific papers objectively is crucial for research output assessment, which subsequently affects institution and country rankings, research funding allocations, academic recruitment and national/international scientific priorities. While most of the assessment schemes based on publication citations may potentially be manipulated through negative citations, in this study, we explore Conflict of Interest (COI) relationships and discover negative citations and subsequently weaken the associated citation strength. PANDORA (Positive And Negative COI- Distinguished Objective Rank Algorithm) has been developed, which captures the positive and negative COI, together with the positive and negative suspected COI relationships. In order to alleviate the influence caused by negative COI relationship, collaboration times, collaboration time span, citation times and citation time span are employed to determine the citing strength; while for positive COI relationship, we regard it as normal citation relationship. Furthermore, we calculate the impact of scholarly papers by PageRank and HITS algorithms, based on a credit allocation algorithm which is utilized to assess the impact of institutions fairly and objectively. Experiments are conducted on the publication dataset from American Physical Society (APS) dataset, and the results demonstrate that our method significantly outperforms the current solutions in Recommendation Intensity of list R at top-K and Spearman's rank correlation coefficient at top-K.