Global assessment of university research comprehensiveness

TL;DR

This paper introduces a new method for assessing university research comprehensiveness by normalizing across fields, revealing that coverage breadth, rather than performance, distinguishes leading institutions, and aligns well with existing rankings.

Contribution

It proposes a non-zero-sum, field-normalized assessment approach that evaluates university comprehensiveness and compares it with traditional league tables, highlighting factors influencing institutional coverage.

Findings

Comprehensive coverage correlates with high rankings.

No correlation between size and quality when size is controlled.

Reputation does not always match research performance.

Abstract

The demand for global university league tables has been high over the past two decades. However, significant criticism of their methodologies is accumulating without being addressed. I revisit global university league tables by normalizing each field as to create a uniform distribution of value. Then, the overall performance of an institution is interpreted as the probability of having a high score in any given academic field. I focus on the similarity of institutions across ten criteria related to academic performance in eighty subjects of all fields of knowledge. The latter does not induce a zero-sum game, removing one of the most prominent negative features of established league tables. The present assessment shows that the main difference between hundreds of leading global research universities is whether their coverage of all areas of human knowledge is comprehensive or specialized, as their mean performance per subject is nearly indistinguishable. I compare the results with the main league tables and found excellent agreement, suggesting that regardless of their methodologies, research-intensive institutions perform well in rankings if they are comprehensive. This comprehensiveness is ultimately dependent on institutional age, privileged funding allocation and regional academic culture. Consequently, when the size of an institution is taken out of the picture, I found no correlation between comprehensiveness and quality, and no difference can be found in the mean quality of institutions regionally or globally. Furthermore, I find the reputation and prestige of several famous institutions to far exceed their performance within the present methodology, while numerous institutions with less reputation and visibility perform better than expected.