The influence of publication ranking specifications on publication strategy and academic careers in business administration

TL;DR

This study shows how different ways of ranking academic publications affect evaluations of business administration researchers and their career outcomes.

Contribution

The study introduces a methodological stress test for ranking systems using multiple variants and real-world data.

Findings

Quantity-focused rankings show high variability and small performance differences among researchers.

Quality-focused rankings consistently identify top researchers who publish more in high-tier, English-language journals.

Ranking design significantly influences academic visibility and career advancement for early-career researchers.

Abstract

This study examines the impact of methodological variations in publication-based rankings on the evaluation of individual research performance in business administration. Drawing on a unique dataset comprising complete personal publication lists of 233 professors from Austrian public universities (2009–2018), we apply ten distinct ranking variants that differ in their treatment of data sources, co-authorship, publication languages, article lengths, and journal qualities. These variants are categorized into purely quantity-focused and predominantly quality-focused rankings. Our results demonstrate that researcher rankings are susceptible to specification choices. While quantity-focused rankings produce relatively small performance differentials and high variability, quality-focused variants consistently identify a stable group of leading researchers. These scholars publish more frequently in English, in journals indexed by Web of Science (WoS), and in top-tier outlets according to the JOURQUAL ranking. Notably, leading researchers publish over twice as many articles in high-ranking journals as their peers. The findings underscore the significant implications of ranking design for career advancement and research strategy. For early-career researchers, aligning publication efforts with the logic of quality-focused rankings—favoring English-language publications in highly ranked, peer-reviewed journals—is crucial for enhancing academic visibility and competitiveness. Moreover, our study offers a methodological stress test for ranking systems, revealing the extent to which technical design influences outcomes. By leveraging comprehensive and multilingual publication data and systematically comparing multiple ranking methodologies, this study contributes to both the academic evaluation literature and practical guidance for researchers navigating the demands of a metric-driven academic environment.

Figures