Usage Impact Factor: the effects of sample characteristics on usage-based impact metrics

TL;DR

This paper introduces a Usage Impact Factor based on local usage data, compares it to the traditional citation-based Impact Factor, and finds that local usage metrics reflect community-specific characteristics rather than global impact.

Contribution

It defines and calculates a Usage Impact Factor from large-scale local usage data and compares it to the traditional Impact Factor to analyze community-specific effects.

Findings

Usage Impact Factor reflects community-specific characteristics.

Local usage data differs from global citation-based metrics.

Community demographics influence impact assessments.

Abstract

There exist ample demonstrations that indicators of scholarly impact analogous to the citation-based ISI Impact Factor can be derived from usage data. However, contrary to the ISI IF which is based on citation data generated by the global community of scholarly authors, so far usage can only be practically recorded at a local level leading to community-specific assessments of scholarly impact that are difficult to generalize to the global scholarly community. We define a journal Usage Impact Factor which mimics the definition of the Thomson Scientific's ISI Impact Factor. Usage Impact Factor rankings are calculated on the basis of a large-scale usage data set recorded for the California State University system from 2003 to 2005. The resulting journal rankings are then compared to Thomson Scientific's ISI Impact Factor which is used as a baseline indicator of general impact. Our results indicate that impact as derived from California State University usage reflects the particular scientific and demographic characteristics of its communities.