Indicators as judgment devices: Citizen bibliometrics in biomedicine and economics

TL;DR

This paper examines how bibliometric indicators serve as judgment devices in evaluating academic candidates in biomedicine and economics, revealing their use in ranking and assessing research impact within Swedish universities.

Contribution

It introduces the concept of citizen bibliometrics, analyzing how indicators are employed in evaluative practices and how expertise in their use is negotiated between domain knowledge and metric skills.

Findings

Bibliometric indicators are extensively used in academic evaluations.

Different metrics are employed explicitly and implicitly to rank candidates.

The use of metrics is negotiated at the interface of domain expertise and indicator skills.

Abstract

The number of publications has been a fundamental merit in the competition for academic positions since the late 18th century. Today, the simple counting of publications has been supplemented with a whole range of bibliometric measures, which supposedly not only measures the volume of research but also its impact. In this study, we investigate how bibliometrics are used for evaluating the impact and quality of publications in two specific settings: biomedicine and economics. Our study exposes the extent and type of metrics used in external evaluations of candidates for academic positions at Swedish universities. Moreover, we show how different bibliometric indicators, both explicitly and implicitly, are employed to value and rank candidates. Our findings contribute to a further understanding of bibliometric indicators as judgment devices that are employed in evaluating individuals and their published works within specific fields. We also show how expertise in using bibliometrics for evaluative purposes is negotiated at the interface between domain knowledge and skills in using indicators. In line with these results we propose that the use of metrics in this context is best described as a form of citizen bibliometrics - an underspecified term which we build upon in the paper.