Interpreting the outcomes of research assessments: a geometrical approach

TL;DR

This paper introduces a geometric method to compare and rank research assessments of academic institutions based on publication quality distributions, providing a new score to evaluate institutional performance objectively.

Contribution

It proposes a novel geometric approach and a geometric score for comparing research assessment outcomes, ensuring fair evaluation of institutions with different profiles.

Findings

Applied the method to rank Italian universities in two scientific areas.

Demonstrated the geometric score's effectiveness in fair comparison.

Provided a new tool for research evaluation analysis.

Abstract

Research evaluations and comparison of the assessments of academic institutions (scientific areas, departments, universities etc.) are among the major issues in recent years in higher education systems. One method, followed by some national evaluation agencies, is to assess the research quality by the evaluation of a limited number of publications in a way that each publication is rated among $n$ classes. This method produces, for each institution, a distribution of the publications in the $n$ classes. In this paper we introduce a natural geometric way to compare these assessments by introducing an ad hoc distance from the performance of an institution to the best possible achievable assessment. Moreover, to avoid the methodological error of comparing non-homogeneous institutions, we introduce a {\em geometric score} based on such a distance. The latter represents the probability that an ideal institution, with the same configuration as the one under evaluation, performs worst. We apply our method, based on the geometric score, to rank, in two specific scientific areas, the Italian universities using the results of the evaluation exercise VQR 2011-2014.