Citation success index - An intuitive pair-wise journal comparison metric

TL;DR

The paper introduces the citation success index, a new metric for comparing journals based on citation distributions, which is less skewed by outliers than impact factor and provides a probabilistic measure of citation advantage.

Contribution

It proposes the citation success index as an intuitive, distribution-aware alternative to impact factor for journal comparison, supported by analysis of 16,000 journals.

Findings

Citation success index correlates with impact factor ratio.

Index exceeds 90% only when IF ratio > 6.

A twofold IF difference yields about 70% success index.

Abstract

In this paper we present "citation success index", a metric for comparing the citation capacity of pairs of journals. Citation success index is the probability that a random paper in one journal has more citations than a random paper in another journal (50% means the two journals do equally well). Unlike the journal impact factor (IF), the citation success index depends on the broadness and the shape of citation distributions. Also, it is insensitive to sporadic highly-cited papers that skew the IF. Nevertheless, we show, based on 16,000 journals containing ~2.4 million articles, that the citation success index is a relatively tight function of the ratio of IFs of journals being compared, due to the fact that journals with same IF have quite similar citation distributions. The citation success index grows slowly as a function of IF ratio. It is substantial (>90%) only when the ratio of IFs exceeds ~6, whereas a factor of two difference in IF values translates into a modest advantage for the journal with higher IF (index of ~70%). We facilitate the wider adoption of this metric by providing an online calculator that takes as input parameters only the IFs of the pair of journals.