End-to-end evaluation of research organizations

TL;DR

This paper proposes a comprehensive framework for evaluating research organizations by separating bibliometric and econometric components, incorporating size-dependent and size-independent indicators for multi-level performance assessment.

Contribution

It introduces a novel end-to-end evaluation methodology that integrates bibliometric indicators with econometric efficiency measures, considering multiple levels of performance and productivity.

Findings

Performance can be evaluated at zeroth, first, or second-order levels.

Size-dependent and size-independent indicators are crucial for meaningful assessment.

Efficiency is represented through output and outcome productivities.

Abstract

End-to-end research evaluation needs to separate out the bibliometric part of the chain from the econometric part. We first introduce the role of size-dependent and size-independent indicators in the bibliometric part of the evaluation chain. We show that performance can then be evaluated at various levels, namely a zeroth-order, a first-order or even a second-order. To complete the evaluation chain, we take up the econometric part where efficiency of the research production process is represented in terms of output and outcome productivities. Both size-dependent and size-independent terms play a crucial role to combine quantity and quality (impact) in a meaningful way. Output or outcome at the bibliometric level can be measured using zeroth, first or second-order composite indicators, and the productivity terms follow accordingly using the input to output or outcome factors.