Introducing recalibrated academic performance indicators in the evaluation of individuals' research performance: A case study from Eastern Europe

TL;DR

This paper proposes recalibrated, discipline-specific academic performance indicators for evaluating research performance, addressing the limitations of uniform thresholds in Hungary's DSc qualification process, with potential applications across Eastern Europe.

Contribution

It introduces a method to adjust performance indicators based on discipline-specific data and counting methods, improving fairness in research evaluations.

Findings

Recalibrated performance thresholds better reflect discipline-specific research output.

Integer and fractional counting methods provide nuanced evaluation metrics.

The approach can be adapted for use in other Eastern European countries.

Abstract

In Hungary, the highest and most prestigious scientific qualification is considered to be the Doctor of Science (DSc) title being awarded by the Hungarian Academy of Sciences. The academic performance indicators of the DSc title are of high importance in the evaluation of individuals' research performance not only when a researcher applies for obtaining a DSc title, but also during promotions and appointments at universities, and in the case of the evaluation of applications for scientific titles and degrees, and the assessment of applications for funding. In the Section of Earth Sciences encompassing nine related disciplines, rather than carrying out a straightforward bibliometric analysis, the performance indicators were designed as a result of a consensual agreement between leading academicians, each of whom represented a particular discipline. Therefore, the minimum values of the indicators, required to be fulfilled if one is applying for a DSc title, do not adequately reflect the actual discipline-specific performance of researchers. This problem may generate tension between researchers during the evaluation process. The main goal of this paper is to recalibrate the minimum values of four major performance indicators by taking the actual discipline-specific distance ratios into account. In addition, each minimum value will be defined by employing integer and fractional counting methods as well. The research outcome of this study can provide impetus for the Section of Earth Sciences to optimize the minimum values of the DSc title performance indicators by taking the specifics of each discipline into account. Because academic performance indicators are also employed in other Eastern European countries in the evaluation of individuals' research performance, the methods used in that paper can be placed into a wider geographical context.