How Can Journal Impact Factors be Normalized across Fields of Science? An Assessment in terms of Percentile Ranks and Fractional Counts

TL;DR

This study evaluates methods like fractional counting and percentile transformation to normalize journal impact factors across scientific fields, finding that longer citation windows improve field normalization.

Contribution

It demonstrates that fractional counting combined with percentile ranks significantly reduces between-field variance in impact measures, especially with five-year impact factors.

Findings

Fractional counting reduces between-field variance by up to 91.7%.

Transforming citation counts into percentiles aids in normalization.

Longer citation windows improve impact factor normalization.

Abstract

Using the CD-ROM version of the Science Citation Index 2010 (N = 3,705 journals), we study the (combined) effects of (i) fractional counting on the impact factor (IF) and (ii) transformation of the skewed citation distributions into a distribution of 100 percentiles and six percentile rank classes (top-1%, top-5%, etc.). Do these approaches lead to field-normalized impact measures for journals? In addition to the two-year IF (IF2), we consider the five-year IF (IF5), the respective numerators of these IFs, and the number of Total Cites, counted both as integers and fractionally. These various indicators are tested against the hypothesis that the classification of journals into 11 broad fields by PatentBoard/National Science Foundation provides statistically significant between-field effects. Using fractional counting the between-field variance is reduced by 91.7% in the case of IF5, and by 79.2% in the case of IF2. However, the differences in citation counts are not significantly affected by fractional counting. These results accord with previous studies, but the longer citation window of a fractionally counted IF5 can lead to significant improvement in the normalization across fields.