Predicting results of the Research Excellence Framework using departmental $h$-Index

TL;DR

This paper evaluates the effectiveness of the departmental h-index in predicting UK research performance, demonstrating it correlates better with peer review results than other metrics and providing unbiased forecasts for the 2014 REF outcomes.

Contribution

The study introduces a departmental h-index-based method for predicting research assessment outcomes, showing its superior correlation with peer review results compared to other bibliometric indicators.

Findings

Departmental h-index correlates better with peer review results than normalized citation metrics.

Predicted REF 2014 results using h-index prior to official publication.

Proposes bibliometric-based predictions as unbiased estimates of research performance.

Abstract

We compare estimates for past institutional research performances coming from two bibliometric indicators to the results of the UK's Research Assessment Exercise which last took place in 2008. We demonstrate that a version of the departmental h-index is better correlated with the actual results of that peer-review exercise than a competing metric known as the normalised citation-based indicator. We then determine the corresponding h-indices for 2008-2013, the period examined in the UK's Research Excellence Framework (REF) 2014. We place herewith the resulting predictions on the arXiv in advance of the REF results being published (December 2014). These may be considered as unbiased predictions of relative performances in that exercise. We will revisit this paper after the REF results are available and comment on the reliability or otherwise of these bibliometrics as compared with peer review.