A Longitudinal Analysis of University Rankings

TL;DR

This study examines the methodologies and stability of major global university rankings, revealing they primarily measure reputation and research, and highlighting discrepancies and critiques of their validity.

Contribution

It provides a detailed analysis of ranking methodologies, their stability over time, and the underlying factors they measure, offering insights into their limitations and implications.

Findings

Rankings are stable over time but vary between sources.

Variables mainly measure reputation and research performance.

Rankings may not fully capture the intended university qualities.

Abstract

Pressured by globalization and the increasing demand for public organisations to be accountable, efficient and transparent, university rankings have become an important tool for assessing the quality of higher education institutions. It is therefore important to carefully assess exactly what these rankings measure. In this paper, the three major global university rankings, The Academic Ranking of World Universities, The Times Higher Education and the Quacquarelli Symonds World University Rankings, are studied. After a description of the ranking methodologies, it is shown that university rankings are stable over time but that there is variation between the three rankings. Furthermore, using Principal Component Analysis and Exploratory Factor Analysis, we show that the variables used to construct the rankings primarily measure two underlying factors: a universities reputation and its research performance. By correlating these factors and plotting regional aggregates of universities on the two factors, differences between the rankings are made visible. Last, we elaborate how the results from these analysis can be viewed in light of often voiced critiques of the ranking process. This indicates that the variables used by the rankings might not capture the concepts they claim to measure. Doing so the study provides evidence of the ambiguous nature of university ranking's quantification of university performance.