Computing a consensus journal meta-ranking using paired comparisons and adaptive lasso estimators

TL;DR

This paper introduces a novel method for creating consensus journal meta-rankings by leveraging paired comparison data and adaptive lasso estimators to identify journal quality clusters and handle missing data.

Contribution

It proposes a parametric model combined with shrinkage techniques to estimate journal quality scores from heterogeneous rankings, addressing data missingness and clustering journals by quality.

Findings

Effective estimation of journal quality scores

Identification of journal clusters with similar quality

Robustness to missing comparison data

Abstract

In a "publish-or-perish culture", the ranking of scientific journals plays a central role in assessing performance in the current research environment. With a wide range of existing methods and approaches to deriving journal rankings, meta-rankings have gained popularity as a means of aggregating different information sources. In this paper, we propose a method to create a consensus meta-ranking using heterogeneous journal rankings. Using a parametric model for paired comparison data we estimate quality scores for 58 journals in the OR/MS community, which together with a shrinkage procedure allows for the identification of clusters of journals with similar quality. The use of paired comparisons provides a flexible framework for deriving a consensus score while eliminating the problem of data missingness.