The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011

TL;DR

The paper introduces a new excellence indicator for SCImago Institutions Rankings that enables statistical significance testing of differences between institutions' research outputs.

Contribution

It presents a novel statistical method and a calculator tool to assess the significance of ranking differences in research excellence indicators.

Findings

The indicator allows testing of whether differences are statistically significant.

The tool helps institutions evaluate their performance relative to expectations.

It accounts for stochastic variations in research output.

Abstract

The new excellence indicator in the World Report of the SCImago Institutions Rankings (SIR) makes it possible to test differences in the ranking in terms of statistical significance. For example, at the 17th position of these rankings, UCLA has an output of 37,994 papers with an excellence indicator of 28.9. Stanford University follows at the 19th position with 37,885 papers and 29.1 excellence, and z = - 0.607. The difference between these two institution thus is not statistically significant. We provide a calculator at http://www.leydesdorff.net/scimago11/scimago11.xls in which one can fill out this test for any two institutions and also for each institution on whether its score is significantly above or below expectation (assuming that 10% of the papers are for stochastic reasons in the top-10% set).