Severe Language Effect in University Rankings: Particularly Germany and France are wronged in citation-based rankings

TL;DR

This paper reveals a significant language bias in citation-based university rankings, especially affecting Germany and France, due to the reliance on English-only publications in bibliometric assessments.

Contribution

It demonstrates the extent of language effects in bibliometric rankings and highlights the need to account for non-English publications in research evaluation.

Findings

Language bias significantly impacts university rankings.

Germany and France are particularly affected.

Citation-based metrics underestimate non-English research output.

Abstract

We applied a set of standard bibliometric indicators to monitor the scientific state-of-arte of 500 universities worldwide and constructed a ranking on the basis of these indicators (Leiden Ranking 2010). We find a dramatic and hitherto largely underestimated language effect in the bibliometric, citation-based measurement of research performance when comparing the ranking based on all Web of Science (WoS) covered publications and on only English WoS covered publications, particularly for Germany and France.