Quantitative Analysis of the Italian National Scientific Qualification

TL;DR

This paper provides a detailed quantitative analysis of Italy's National Scientific Qualification process, highlighting disciplinary differences, issues with bibliometric indicators, and implications for future evaluations.

Contribution

It offers the first comprehensive quantitative assessment of the ASN, revealing disparities among disciplines and raising concerns about indicator definitions and thresholds.

Findings

Significant disciplinary differences in qualification rates.

Issues with bibliometric indicator thresholds.

Potential biases affecting qualification outcomes.

Abstract

The Italian National Scientific Qualification (ASN) was introduced in 2010 as part of a major reform of the national university system. Under the new regulation, the scientific qualification for a specific role (associate or full professor) and field of study is required to apply to a permanent professor position. The ASN is peculiar since it makes use of bibliometric indicators with associated thresholds as one of the parameters used to assess applicants. Overall, more than 59000 applications were submitted, and the results have been made publicly available for a short period of time, including the values of the quantitative indicators for each applicant. The availability of this wealth of information provides an opportunity to draw a fairly detailed picture of a nation-wide evaluation exercise, and to study the impact of the bibliometric indicators on the qualification results. In this paper we provide a first account of the Italian ASN from a quantitative point of view. We show that significant differences exist among scientific disciplines, in particular with respect to the fraction of qualified applicants, that can not be easily explained. Furthermore, we describe some issues related to the definition and use of the bibliometric indicators and thresholds. Our analysis aims at drawing attention to potential problems that should be addressed by decision-makers in future ASN rounds.