Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA
Loet Leydesdorff, Caroline S. Wagner, and Lin Zhang

TL;DR
This paper investigates whether differences in university rankings are statistically significant using z-tests and confidence intervals, comparing Chinese and US universities, and discusses implications for policy and ranking interpretation.
Contribution
It introduces statistical methods to assess the significance of ranking differences and applies them to Chinese and US universities, highlighting the importance of significance testing in ranking analysis.
Findings
Chinese and US universities form three main groups: high, middle, low.
Chinese universities' z-scores are significantly below US universities, with some overlap.
Ranking differences can be due to data, models, or methodological effects.
Abstract
Purpose: We address the question of whether differences are statistically significant in the rankings of universities. We propose methods measuring the statistical significance among different universities and illustrate the results by empirical data. Design/methodology/approach: Based on z-testing and overlapping confidence intervals, and using data about 205 Chinese universities included in the Leiden Rankings 2020, we argue that three main groups of Chinese research universities can be distinguished. Findings: When the sample of 205 Chinese universities is merged with the 197 US universities included in Leiden Rankings 2020, the results similarly indicate three main groups: high, middle, low. Using this data (Leiden Rankings and Web-of-Science), the z-scores of the Chinese universities are significantly below those of the US universities albeit with some overlap. Research…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Topicsscientometrics and bibliometrics research
