A critical cluster analysis of 44 indicators of author-level performance

TL;DR

This study uses cluster analysis to evaluate the disciplinary and seniority relevance of 44 author performance indicators across four disciplines, revealing discipline-specific indicators and a weak link between CVs and bibliometric scores.

Contribution

It introduces a cluster-based approach to assess the appropriateness of bibliometric indicators for different disciplines and seniorities, highlighting the need for contextual validation.

Findings

Different indicators are suitable for ranking performance in different disciplines.

Researcher performance is mainly influenced by years since first publication, number of publications, and citations.

Seniority classification has limited impact on indicator appropriateness.

Abstract

This paper explores the relationship between author-level bibliometric indicators and the researchers the "measure", exemplified across five academic seniorities and four disciplines. Using cluster methodology, the disciplinary and seniority appropriateness of author-level indicators is examined. Publication and citation data for 741 researchers across Astronomy, Environmental Science, Philosophy and Public Health was collected in Web of Science (WoS). Forty-four indicators of individual performance were computed using the data. A two-step cluster analysis using IBM SPSS version 22 was performed, followed by a risk analysis and ordinal logistic regression to explore cluster membership. Indicator scores were contextualized using the individual researcher's curriculum vitae. Four different clusters based on indicator scores ranked researchers as low, middle, high and extremely high performers. The results show that different indicators were appropriate in demarcating ranked performance in different disciplines. In Astronomy the h2 indicator, sum pp top prop in Environmental Science, Q2 in Philosophy and e-index in Public Health. The regression and odds analysis showed individual level indicator scores were primarily dependent on the number of years since the researcher's first publication registered in WoS, number of publications and number of citations. Seniority classification was secondary therefore no seniority appropriate indicators were confidently identified. Cluster methodology proved useful in identifying disciplinary appropriate indicators providing the preliminary data preparation was thorough but needed to be supplemented by other analyses to validate the results. A general disconnection between the performance of the researcher on their curriculum vitae and the performance of the researcher based on bibliometric indicators was observed.