A principal component analysis of 39 scientific impact measures

TL;DR

This paper uses principal component analysis to explore how 39 different measures of scientific impact, including traditional citations and new online activity metrics, relate to each other and capture impact.

Contribution

It provides a comprehensive analysis of various impact measures, revealing that scientific impact is multi-dimensional and that traditional metrics like Impact Factor are peripheral.

Findings

Impact is multi-dimensional and cannot be captured by a single measure.

Some impact measures are more representative of scientific impact than others.

Impact Factor is at the periphery of the impact construct, not at its core.

Abstract

The impact of scientific publications has traditionally been expressed in terms of citation counts. However, scientific activity has moved online over the past decade. To better capture scientific impact in the digital era, a variety of new impact measures has been proposed on the basis of social network analysis and usage log data. Here we investigate how these new measures relate to each other, and how accurately and completely they express scientific impact. We performed a principal component analysis of the rankings produced by 39 existing and proposed measures of scholarly impact that were calculated on the basis of both citation and usage log data. Our results indicate that the notion of scientific impact is a multi-dimensional construct that can not be adequately measured by any single indicator, although some measures are more suitable than others. The commonly used citation Impact Factor is not positioned at the core of this construct, but at its periphery, and should thus be used with caution.