Analyzing the disciplinary focus of universities: Can rankings be a one-size-fits-all?

TL;DR

This paper critically examines how university rankings oversimplify research outputs by ignoring disciplinary differences, which can lead to misinterpretation and poor decision-making in higher education evaluation.

Contribution

It provides a methodological analysis of how rankings suppress disciplinary distinctions, highlighting the limitations of current bibliometric-based evaluation tools.

Findings

Rankings often ignore disciplinary differences in research output.

Disciplinary specialization significantly influences research evaluation.

Current rankings may lead to misinterpretation of university performance.

Abstract

The phenomenon of rankings is intimately related with the government interest in fiscalizing the research outputs of universities. New forms of managerialism have been introduced into the higher education system, leading to an increasing interest from funding bodies in developing external evaluation tools to allocate funds. Rankings rely heavily on bibliometric indicators. But bibliometricians have been very critical with their use. Among other, they have pointed out the over-simplistic view rankings represent when analyzing the research output of universities, as they consider them as homogeneous ignoring disciplinary differences. Although many university rankings now include league tables by fields, reducing the complex framework of universities' research activity to a single dimension leads to poor judgment and decision making. This is partly because of the influence disciplinary specialization has on research evaluation. This chapter analyzes from a methodological perspective how rankings suppress disciplinary differences which are key factors to interpret correctly these rankings.