Academic research groups: evaluation of their quality and quality of their evaluation

TL;DR

This paper critically examines how academic research group quality is evaluated, highlighting issues with current metrics and emphasizing the need for scientifically grounded assessment methods.

Contribution

It provides a clear definition of critical mass, analyzes how group size affects research quality, and critiques the reliability of existing evaluation metrics.

Findings

Group size influences research quality

Current metrics often fail as reliable indicators

Properly defined critical mass impacts evaluation accuracy

Abstract

In recent years, evaluation of the quality of academic research has become an increasingly important and influential business. It determines, often to a large extent, the amount of research funding flowing into universities and similar institutes from governmental agencies and it impacts upon academic careers. Policy makers are becoming increasingly reliant upon, and influenced by, the outcomes of such evaluations. In response, university managers are increasingly attracted to simple indicators as guides to the dynamics of the positions of their various institutions in league tables. However, these league tables are frequently drawn up by inexpert bodies such as newspapers and magazines, using rather arbitrary measures and criteria. Terms such as "critical mass' and "metrics" are often bandied about without proper understanding of what they actually mean. Rather than accepting the rise and fall of universities, departments and individuals on a turbulent sea of arbitrary measures, we suggest it is incumbent upon the scientific community itself to clarify their nature. Here we report on recent attempts to do that by properly defining critical mass and showing how group size influences research quality. We also examine currently predominant metrics and show that these fail as reliable indicators of group research quality.